PERSPECTIVE
Drug Evaluation After Marketing DENNIS SLONE, M.D.; SAMUEL SHAPIRO, M.B.; OLLI S. MIETTINEN, M.D., Ph.D.; WILLIAM D. FINKLE, Ph.D.; and PAUL D. STOLLEY, M.D.; Boston and Cambridge, Massachusetts; and Philadelphia, Pennsylvania After marketing, drugs should be evaluated for safety and for efficacy. Present national systems are inadequate for both tasks. Safety can be evaluated by means of nonexperimental research, whereas evaluation of efficacy in a variety of settings representing normal medical practice generally requires experiments, randomized and blinded. Research into safety requires the maintenance of routine, yet flexible, multipurpose data systems, different for established and newly marketed drugs. Ad hoc studies on safety and on efficacy can be mounted more swiftly and economically through the maintenance of "standby" capabilities, in addition to maximal use of resources outside the actual national system. T H E I N T R O D U C T I O N of any new drug for use in the Unit-
ed States is a matter of increasing social concern and is regulated by the Food and Drug Administration ( F D A ) (1). As part of its regulatory function the F D A has to ensure that a drug is shown to be safe and effective. Studies are carried out according to a specified sequence. The first phase is an evaluation of the drug's pharmacology in animals or tissue preparations (phase 1). This is followed by studies of acute and chronic toxicity in human volunteers (phase 2) and then by clinical trials (usually randomized, double-blind studies) in selected human populations (phase 3). In the phase 3 studies the drug must be shown not only to be safe, but also effective in terms of some particular indication. Generally, it is unrealistic to expect complete safety, and the decision to permit a drug's release to the market must be based on the expectation that the benefit outweighs the risk. Despite the precautions taken to ensure that only drugs with acceptable balances of risk and benefit are released, unsuspected side-effects can be encountered subsequently. If such side-effects are serious, they may necessitate modification of the drug's use or even its withdrawal (as happened in Europe with practolol, a beta receptor blocking agent that was found to cause blindness and sclerosing peritonitis [2]). Occasionally, a side-effect (whether known or unknown at the time of marketing) sometimes leads to a new indication for a drug's use (as happened with propranolol, another beta blocker: T h e side-effect, hypotension, led to its use as an antihypertensive [3]). • From the Drug Epidemiology Unit, Boston University School of Medicine; Cambridge, Massachusetts; the Department of Epidemiology, Harvard School of Public Health; Boston, Massachusetts; and the Department of Research Medicine, University of Pennsylvania School of Medicine; Philadelphia, Pennsylvania.
Annals of Internal Medicine 90:257-261, 1979
Downloaded From: http://annals.org/ by a University of Otago User on 12/09/2016
The discovery of side-effects after marketing used to be quite haphazard and based on such sources as case-reports, journal articles, and letters to the editor. It has been realized more recently that if the F D A is to fulfill its function properly, a national system of drug surveillance will be needed. Over the years, various steps have been taken to implement post-marketing surveillance; but insofar as any system can now be said to exist (4) it is incomplete, as shown by the continuing discovery, outside of any surveillance mechanism, of side-effects of marketed drugs. Although the primary social concern has been with the safety of drugs after marketing, there is also the concern that marketed drugs should be effective, not only in the circumstances of phase 3 studies, but in everyday clinical practice. This concern is accentuated when drugs are intended for long-term use (such as cholesterol-lowering agents) and when drugs come to be used for new indications after marketing (such as propranolol for hypertension). In this paper we seek to identify, as a basis for formulating policy, the central issues pertinent to a national system for the evaluation of drugs for both safety and efficacy after marketing. Surveillance for Safety N E E D FOR S U R V E I L L A N C E
When a decision to permit the marketing of a new drug is made, there are several reasons why the F D A has only incomplete knowledge of its side-effects. Among them are the following. First, phase 3 studies are necessarily of limited size (usually 700 to 3000 subjects in total). Adverse reactions that are uncommon (say, less than 1 per 1000 drug users) may not be detected. Second, many types of patients who will ultimately use the drug are excluded from phase 3 studies (such as persons in certain age groups, pregnant women, patients with diseases other than the one being studied, and patients concomitantly using other medications). This does not permit identification of side-effects that may occur only in certain subgroups of patients or effects which may arise because one drug modifies the action of another. Third, the duration of phase 3 studies tends to be quite limited, usually to 1 to 2 years. This does not permit identification of toxicity that manifests only after long ©1979 American College of Physicians
257
periods of use (such as chloroquine retinopathy [5]) or of side-effects appearing long after first exposure (such as adenocarcinoma of the vagina due to diethylstilbestrol [6]). From these considerations it follows that reassurance about a drug's safety requires long-term surveillance after marketing. Certain types of surveillance have already been in existence for some time (7), but the continuing discovery of important and previously unknown side-effects of drugs that have been available for many years (such as adenocarcinoma of the endometrium due to conjugated estrogens [8]) shows that the surveillance strategies so far implemented can fall short of providing the needed information with minimal delay. It is thus necessary to consider alternative or supplementary strategies that can serve as an adequate national system of drug surveillance. Such a system needs to cover effects of drugs in the fetus and other special populations. It should also be able to document effects of long-term and short-term drug use and effects of varying degrees of severity, and it should be able to identify effects that may only appear after long delays. To these needs a further requirement should be added: Even when an adverse effect is known to occur (whether from premarketing or postmarketing observations) only a proper quantitative estimate of its occurrence can form an adequate basis for therapeutic and regulatory decisions. A surveillance system should provide for such estimation. Furthermore, in carrying out this quantitative role, the system must be able to evaluate possible modification of adverse drug effects by factors such as various patient characteristics (such as age), concomitant drugs, and other factors (for example, the risk of myocardial infarction in women who use oral contraceptives seems to be augmented by cigarette smoking [9]). Taken together, the requirements for a comprehensive national surveillance system are quite formidable, and it is unlikely that they will ever be fulfilled completely. It seems generally to be agreed, however, that improvement of what already exists is possible and desirable. To quote Lasagna (4), "The conferees agreed that the question of whether or not post-marketing surveillance should be done is idle; that a relatively unstructured (open) system of post-marketing surveillance is now in existence; that post-marketing surveillance is an absolute necessity for assessing the benefit/risk ratio of a new drug; and that the proper object of discussion is the question of whether the present system should give way to some of the new (structured) proposals now under discussion, and to what extent."
OBJECTIVES OF S U R V E I L L A N C E
In view of the existing needs, the postmarketing surveillance of drugs for safety should serve three broad objectives: first, the discovery of side-effects unknown at the time of marketing, including those that may subsequently lead to new indications for drug use; second, confirmation of side-effects reported from other sources; and third, the quantitative evaluation of the risks of known 258
adverse effects, whether discovered before or after marketing. S U R V E I L L A N C E SYSTEM COMPONENTS
To meet these objectives, the F D A needs access to information on drug experience that is both timely and of good scientific quality. The attainment of this ideal requires a national system of drug surveillance. This has been recognized for a long time. In 1966, Finney (10) stated that " . . . the monitoring of adverse reactions must become part of the regular framework of health services . . . it is a response to a need to replace haphazard collection and interpretation of information by something more organized . . . " This view was supported in 1970 by the International Conference on Adverse Reaction Reporting Systems (11): "A major conclusion . . . was that the only protection . . . is in an effectively organized system for detecting adverse reactions . . . " Such a national system involves various components fulfilling different needs. Routine Data Systems: Effective surveillance requires systematic recording of drug experience. Two major types of routine data systems are currently in existence. First, there are nationwide programs that register putative adverse reactions; they rely on informal, ad hoc judgments of causality on the part of the reporters. Examples are the existing adverse reaction reporting system maintained by the F D A and the industry-based monitoring of drug experience. Such registries can be used to raise hypotheses of adverse effects. Such use involves the collation and evaluation of putative case reports sent directly to a registry or appearing in medical journals. Beyond the existing adverse reaction reporting schemes, other possibilities remain to be explored. Second, there are formal government-sponsored epidemiologic programs of drug surveillance, each concerned with a limited segment of drug experience (such as the Boston Collaborative Drug Surveillance Program [12], the Drug Epidemiology Unit [13], and the Pediatric Drug Surveillance Program [14]). These systems differ from the registries by using routine accumulation of primary data and formal epidemiologic analyses. In these analyses connections between drugs and side-effects are inferred in the aggregate, and not in individual patients. At the first level of analysis, the identified associations constitute no more than "signals" of possible causal connections. However, beyond simply detecting a potential cause-and-effect relation, the formal epidemiologic systems sometimes permit detailed scientific evaluation of an initial finding (that is, they can allow for the next step of ad hoc research seeking to verify the finding, explore it in detail, and evaluate it in quantitative terms). In any particular instance the ad hoc capability of a surveillance program depends on the nature of the possible connection and the types of data recorded. Capability for Ad Hoc Research: The evaluation of hypotheses, whatever their origin, would be enhanced by the maintenance of clinical-epidemiologic capabilities for the rapid and efficient conduct of various specific studies. Such projects may be prompted by case reports or articles, by associations discovered in the course of routine
February 1979 • Annals of Internal Medicine • Volume 90 • Number 2
Downloaded From: http://annals.org/ by a University of Otago User on 12/09/2016
surveillance, or by suspicions arising from some other source. These capabilities could be extensions of routine surveillance systems, supplementing their activities. The essential features of such "institutionalized capabilities" would be their rapid responsiveness to need, together with economy. This would be achieved by the maintenance of both the access to sources of the requisite new data, and the personnel and facilities to carry out the studies. Marshalling Extramural Resources: In instances of special need for ad hoc studies, effective use should be made of the various scientific resources that exist independently of the national system of surveillance: Certain research groups may have access to particularly useful data collected for other purposes, and they may be able to carry out the needed studies most expeditiously. For example, the most detailed study showing that oral contraceptives appear to cause benign liver tumors made use of an Army pathology registry (15). Effective use of such resources may call for national organization if they are to be mobilized speedily when needed—an aspect of an overall national system. Laboratory Research: The broad postmarketing experience with a drug may be complemented usefully by the maintenance of laboratory and other facilities to explore the more detailed nature, including mechanisms, of putative newly detected adverse drug effects. Conversely, laboratory discoveries may point to the need for ad hoc epidemiologic studies. RELATIONS OF COMPONENTS TO OBJECTIVES
In its broadest outline, a national system would be within the rubric of a national coordinating center having access to various research resources, available to be applied as appropriate. The roles of the components would vary according to whether the objective is one of discovery, verification, or quantitative evaluation. Discovery: All of the components referred to above can play a role in discovery. If an adverse event is acute and rare in the absence of drug use (such as sudden deafness), the discovery of its acute induction by a drug (for example, furosemide [16]) can be accomplished readily by clinical observation. If, on the other hand, the event is quite common (as is the case, for example, with many cancers), its discovery tends to depend on formal epidemiologic studies, whether carried out within the context of surveillance or on an ad hoc basis. When there is a long delay in the appearance of the adverse effect (such as adenocarcinoma of the uterus due to conjugated estrogens [8]), the need for an epidemiologic approach becomes even more compelling. Finally, laboratory research in animals may first suggest the possibility that a drug is causing some adverse effect. Verification of Reported Side-Effects: The choice of the appropriate strategy would depend on the nature of the problem. A hypothesis proposed elsewhere can be tested with existing surveillance data (17) or with ad hoc studies. Such a study is needed, for example, when case reports implicate drugs in the causation of very rare effects
(for example, benign liver tumors due to oral contraceptives [18]). Such effects are too uncommon to be represented in sufficient numbers in routine data systems. Quantitative Evaluation: Sometimes ad hoc research with existing surveillance-generated data bases can be used for quantitative evaluation. The alternatives are ad hoc research carried out using facilities sponsored by a national surveillance system or ad hoc research based on the marshalling of extramural resources. For quantitative evaluation there would not be a role for registries for adverse-reaction reports or for laboratory research concerned with mechanisms. S U R V E I L L A N C E OF N E W L Y M A R K E T E D D R U G S
A policy question of particular concern to the F D A is the organization of a system of surveillance to ensure that unrecognized risks associated with newly marketed drugs are detected as soon as possible after their release (7). In general, such surveillance would entail follow-up of a cohort of the earliest patients treated with the new drug. Appropriate nonexposed cohorts may also be needed for discovery and quantitation of effects. The efficiency of this approach depends on the frequency of the adverse events in those not exposed and on the magnitude of the effect caused by the drug. As yet, a routine system for the formation of such cohorts does not exist, although some ad hoc studies of new drugs have been carried out and have yielded worthwhile results (19). There remains the problem of potential serious adverse effects that are too rare or delayed to be detected in any cohort study of practicable size and duration. A case in point is the alleged connection between the use of cimetidine and bone marrow depression (20). Surveillance for such side-effects would entail monitoring various rare illnesses for changes in incidence, and the use of case-control studies. For both, the central system element would be a registry based upon a very large population. The Commission on Professional and Hospital Activities for example, records approximately 40% of United States hospital diagnoses, and already collaborates with the Center for Disease Control in monitoring congenital malformations (21). Efficacy Evaluation
In the past, the predominant social concern has been with potential side-effects of marketed drugs. However, there are policy issues related to efficacy that also merit consideration. N E E D FOR E V A L U A T I O N
Phase 3 studies of efficacy frequently require supplementation after marketing. The premarketing studies do not document long-term efficacy, efficacy in the context of everyday clinical practice, or efficacy in terms of any new indication for use suggested after marketing. Long-Term Efficacy: The deficiency of the phase 3 studies in the assessment of long-term efficacy derives from their short duration: They seldom last more than 1 or 2 years. Thus, while they can show, for example, that hypoglycemic agents lower blood glucose levels, they do Sloneetal. • Drug Evaluation After Marketing
Downloaded From: http://annals.org/ by a University of Otago User on 12/09/2016
259
not show that such drugs reduce the risk of diabetic complications (22). Efficacy in Clinical Practice: The phase 3 studies tend not to be representative of clinical practice in that they involve rigid adherence to treatment regimens, restricted eligibility for therapy, and other limitations. After marketing, persons who would have been excluded from the premarketing studies are likely to receive the drug, and it may be used in ways quite different from those in a clinical trial (such as dosage, adherence, concomitant drugs). New Indications: As already mentioned, a side-effect (whether identified in premarketing studies or only after marketing) can become a new indication. Alternatively, it may simply be realized, after marketing, that a drug has a previously unrecognized indication (such as phenytoin for the treatment of arrhythmias [23]). Whatever the basis for the suggested new indication, a test of its validity may require that the proposed new indication be subjected to rigorous evaluation analogous to that required in phase 3 studies. In short, the generalizability and relevance of phase 3 studies to the setting in which drugs ultimately come to be used can be severely limited.
E L E M E N T S R E Q U I R E D FOR T H E E V A L U A T I O N OF EFFICACY AFTER MARKETING
The nonexperimental methods used in epidemiologic research are of little value in efficacy research, and clinical trials are usually needed. The reason is that patients generally are selected for therapy with a given drug in such a way that they may differ from those not receiving the drug in their expected clinical course. Thus the indication for drug use is commonly a confounding factor in efficacy research. Often, this is a situation in which the problem of bias due to confounding cannot be handled using nonexperimental methods: usually it is not feasible to record all reasons for prescribing a particular drug. This issue of confounding is sufficiently important and subtle to merit consideration of a detailed example. Suppose that in a randomized double-blind trial propranolol reduces the risk of sudden death among survivors of myocardial infarction. A nonexperimental epidemiologic study might well produce a contrary (and invalid) result because of confounding by the indication for therapy: It is likely that the drug will be prescribed to those thought to have the highest risk of sudden death (inferred, for example, from persistent ventricular arrhythmias). Consequently, the propranolol recipients might tend to show a higher rate of sudden death than the nonrecipients, even though, in fact, the drug reduces the risk. Moreover, the particulars of arrhythmia and other factors leading to the use of this drug (and to adherence to therapy), in any given patient, tend to be too complex and subtle to be fully recorded. This makes detailed allowance for confounding by indication infeasible in the analysis of nonexperimental data. Because of uncontrollable confounding by indication in many, if not most, instances, nonexperimental epidemiologic studies will not suffice. Usually, there will be a need 260
for conducting randomized trials. In these trials, due consideration should be given to attaining some resemblance to the way the drug is used in actual practice. ORGANIZATIONAL FRAMEWORK
As with surveillance for side-effects it may be necessary to implement efficacy evaluation in the postmarketing phase as a nationally coordinated effort. Long-Term Efficacy: That there is need for randomized trials to assess long-term efficacy is borne out by experience. On the one hand, such trials have served to dispel misgivings about a drug's value. Outstanding examples are the Veterans Administration studies showing that therapy for hypertension reduces the risk of stroke (24, 25)). On the other hand, some trials have indicated the need for caution (for example, the finding that high doses of conjugated estrogens given to survivors of myocardial infarction increase, rather than diminish, the risk of subsequent infarction [26]). Although there is no great difficulty in showing that a drug reduces, say, cholesterol levels, the type of clinical trial needed to show that it reduces the risk of ischemic heart disease is technically complex. For such a trial to be practical at all requires a large organization having a coordinating center and interactions with collaborating institutions (27). From the policy perspective of having a national system, consideration should be given to the extent to which any center should be maintained on an institutionalized basis; and there may be a need for more than one center, with each specializing in different types of efficacy studies. Efficacy in Everyday Practice: It is something of a paradox that the very rigor of the orthodox clinical trial limits the generalizability of its findings; and the methods needed to achieve generalizability by carrying out clinical trials in the ordinary clinical setting have hardly been explored. Some tentative explorations of short-term efficacy, in a routine hospital ward setting, have been made with encouraging results (28, 29). This approach merits further exploration. With regard to the evaluation of the long-term efficacy of any drug, as used in everyday practice, no research has been done. Possibly, existing health maintenance organizations might be used to assign drugs on a randomized, double-blind basis to evaluate their long-term efficacy in practice. Evaluation for New Indications: As already pointed out, suggested new indications generally require evaluation in a clinical trial. Whether a new indication should be assessed by a research group responsible for the postmarketing evaluation of drug efficacy or by the manufacturer would to some extent depend on the nature of the indication itself, as well as the nature of the drug. ACKNOWLEDGMENTS: Many of the ideas expressed by the authors were developed in the course of their collaboration, as consultants to IMS America, Ltd., in preparing a Food and Drug Administration-sponsored report (7) on the problems of postmarketing surveillance. The authors are solely responsible, however, for the ideas expressed here. • Requests for reprints should be addressed to Dennis Slone, M.D.; Drug Epidemiology Unit, 10 Moulton Street; Cambridge, MA 02138. Received 27 November
February 1979 • Annals of Internal Medicine • Volume 90 • Number 2
Downloaded From: http://annals.org/ by a University of Otago User on 12/09/2016
1978; accepted 28 November
1978.
References
15. BOURNEROOKS J, O R Y HW, ISHAK K G , STRAUSS LT, G R E E N S P A N JR,
1. Interim Report. Expansion of FDA's statutory authority in the postmarketing period for new drugs. Review Panel on New Drug Regulation (DPHEW), Rockville, Maryland, 1977. 2. Hazards of non-practolol beta-blockers (editorial). Br Med J 1:529-530, 1977
TYLER CW: The association between oral contraception and hepato-cellular adenoma—a preliminary report. Int J Gynecol Obstet 15:143-149, 1977 16. COOPERMAN LB, RUBIN IL: Toxicity of ethacrynic acid and furosemide. Am Heart J 85:831-834, 1973 17. BOSTON COLLABORATIVE D R U G S U R V E I L L A N C E P R O G R A M M E : Oral
contraceptives and venous thromboembolic disease, surgically confirmed gallbladder disease, and breast tumours. Lancet 1:1399-1404, 1973
3. ZACHARIAS FJ, C O W E N KJ, PRESST J, VICKERS J, W A L L BG: Propra-
4. 5. 6. 7.
8.
nolol in hypertension: a study of long-term therapy, 1964-1970. Am Heart 7 83:755-761, 1972 LASAGNA L: Post-Marketing Surveillance of Drugs. Washington, D.C., Medicine in the Public Interest, Inc., 1976, p. 6 BERNSTEIN HN: Chloroquine ocular toxicity. Surv Ophthalmol 12:415447, 1967 DES Task force summary report. Part II: DES Daughters (DHEW publication no. (NIH) 79-1688), Bethesda, Maryland, 1978 IMS AMERICA LTD.: Health Care Services Research Group: Final Report—task A. An Experiment in Early Post-Marketing Surveillance of Drugs. ( D P H E W / P H S / F D A . Rockville, Maryland), 1978 WEISS NS: Interpretation of the association between exogenous estrogens and endometrial cancer, in Epidemiological Evaluation of Drugs,
between benign hepatomas and oral contraceptives. Lancet 2:926-929, 1973 19. Azaribine (triazure) recalled. FDA Drug Bull 6:34, 1976 20.
D E GALOCSY C, VAN Y P E R S E L E D E STRIHOU C: Pancytopenia with
edited by C O L O M B O F, SHAPIRO S, SLONE D, T O G N O N I G. Littleton,
cimetidine (letter). Ann Intern Med 90:214, 1979 21. The birth defects monitoring program. Am Coll Obstet Gynecol Newslet, February 1974 22. The University Group Diabetes Program. A study of the effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes. II. Mortality results. Diabetes 19: (Suppl 2), 787-830, 1970 23. HARRIS AS, KOKERNOT RH: Effects of diphenylhydantoin sodium and phenobarbital sodium upon ectopic ventricular tachycardia in acute myocardial infarction. Am J Physiol 163:505-516, 1950
Massachusetts, Publishing Sciences Group, Inc., 1977, pp. 161-167
24. V E T E R A N S A D M I N I S T R A T I O N C O O P E R A T I V E STUDY G R O U P ON A N -
9. M A N N JI, D O L L R, T H O R O G O O D M, VESSEY MP, W A T E R S WE: Risk
factors for myocardial infarction in young women. Br J Prev Soc Med 30:94-100, 1976 10. FINNEY DJ: Monitoring adverse reactions to drugs—its logic and its weaknesses, in Proceedings of the European Society for the Study of Drug Toxicity. Amsterdam, Exerpta Medica Foundation, 1966, p. 198 11. Report of the International Conference on Adverse Reaction Reporting Systems. Washington, D.C., National Academy of Sciences, 1971, p. 5 12.
18. B A U M JK, H O L T Z F, BOOKSTEIN JJ, K L E I N EW: Possible association
JlCK H, MlETTINEN O S , SHAPIRO S, LEWIS G P , SlSKIND V, SLONE D :
Comprehensive drug surveillance. JAMA
213:1455-1460, 1970
TIHYPERTENSIVE AGENTS: Effects of treatment on morbidity in hypertension: results in patients with diastolic blood pressure averaging 115 through 129 mm Hg. JAMA 202:1028-1034, 1967 25. V E T E R A N S A D M I N I S T R A T I O N C O O P E R A T I V E STUDY G R O U P ON A N -
TIHYPERTENSIVE AGENTS: Effects of treatment on morbidity in hypertension. II. Results in patients with diastolic blood pressure averaging 90 through 114 mm Hg. JAMA 213:1143-1152, 1970 26. T H E CORONARY D R U G P R O J E C T R E S E A R C H G R O U P : The Coronary
Drug Project. Findings leading to discontinuation of the 2.5-mg/day estrogen group. JAMA 226:652-657, 1973
13. SLONE D, SHAPIRO S, M I E T T I N E N OS: Case-control surveillance of seri-
27. T H E CORONARY D R U G P R O J E C T R E S E A R C H G R O U P : The Coronary
ous illnesses attributable to ambulatory drug use. See reference 8, pp. 5970
Drug Project. Design, methods, and baseline results. Circulation 47: (Suppl 1), 1973
14. M I T C H E L L AA, G O L D M A N P, SHAPIRO S, SISKIND V, SLONE D: Inten-
28. JICK H, SLONE D, SHAPIRO S, L E W I S G P , SISKIND V: A new method
sive drug surveillance in a pediatric hospital, in Clinical Pharmacy and Clinical Pharmacology, edited by GOUVEIA WA, TOGNONI A, V A N D E R KLEIJN E. Amsterdam, Elsevier/North Holland Biomedical Press, 1976, pp. 303-310
for assessing the clinical effects of oral analgesic drugs. Clin Pharmacol Ther 12:456-463, 1971 29. JICK H, SLONE D, SHAPIRO S, L E W I S G P : Clinical effects of hypnotics.
I. A controlled trial. JAMA
Slone etal.
Downloaded From: http://annals.org/ by a University of Otago User on 12/09/2016
209:2013-2015, 1969
• Drug Evaluation After Marketing
261
Drug Evaluation After Marketing DENNIS SLONE, M.D.; SAMUEL SHAPIRO, M.B.; OLLI S. MIETTINEN, M.D., Ph.D.; WILLIAM D. FINKLE, Ph.D.; and PAUL D. STOLLEY, M.D.; Boston and Cambridge, Massachusetts; and Philadelphia, Pennsylvania After marketing, drugs should be evaluated for safety and for efficacy. Present national systems are inadequate for both tasks. Safety can be evaluated by means of nonexperimental research, whereas evaluation of efficacy in a variety of settings representing normal medical practice generally requires experiments, randomized and blinded. Research into safety requires the maintenance of routine, yet flexible, multipurpose data systems, different for established and newly marketed drugs. Ad hoc studies on safety and on efficacy can be mounted more swiftly and economically through the maintenance of "standby" capabilities, in addition to maximal use of resources outside the actual national system. T H E I N T R O D U C T I O N of any new drug for use in the Unit-
ed States is a matter of increasing social concern and is regulated by the Food and Drug Administration ( F D A ) (1). As part of its regulatory function the F D A has to ensure that a drug is shown to be safe and effective. Studies are carried out according to a specified sequence. The first phase is an evaluation of the drug's pharmacology in animals or tissue preparations (phase 1). This is followed by studies of acute and chronic toxicity in human volunteers (phase 2) and then by clinical trials (usually randomized, double-blind studies) in selected human populations (phase 3). In the phase 3 studies the drug must be shown not only to be safe, but also effective in terms of some particular indication. Generally, it is unrealistic to expect complete safety, and the decision to permit a drug's release to the market must be based on the expectation that the benefit outweighs the risk. Despite the precautions taken to ensure that only drugs with acceptable balances of risk and benefit are released, unsuspected side-effects can be encountered subsequently. If such side-effects are serious, they may necessitate modification of the drug's use or even its withdrawal (as happened in Europe with practolol, a beta receptor blocking agent that was found to cause blindness and sclerosing peritonitis [2]). Occasionally, a side-effect (whether known or unknown at the time of marketing) sometimes leads to a new indication for a drug's use (as happened with propranolol, another beta blocker: T h e side-effect, hypotension, led to its use as an antihypertensive [3]). • From the Drug Epidemiology Unit, Boston University School of Medicine; Cambridge, Massachusetts; the Department of Epidemiology, Harvard School of Public Health; Boston, Massachusetts; and the Department of Research Medicine, University of Pennsylvania School of Medicine; Philadelphia, Pennsylvania.
Annals of Internal Medicine 90:257-261, 1979
Downloaded From: http://annals.org/ by a University of Otago User on 12/09/2016
The discovery of side-effects after marketing used to be quite haphazard and based on such sources as case-reports, journal articles, and letters to the editor. It has been realized more recently that if the F D A is to fulfill its function properly, a national system of drug surveillance will be needed. Over the years, various steps have been taken to implement post-marketing surveillance; but insofar as any system can now be said to exist (4) it is incomplete, as shown by the continuing discovery, outside of any surveillance mechanism, of side-effects of marketed drugs. Although the primary social concern has been with the safety of drugs after marketing, there is also the concern that marketed drugs should be effective, not only in the circumstances of phase 3 studies, but in everyday clinical practice. This concern is accentuated when drugs are intended for long-term use (such as cholesterol-lowering agents) and when drugs come to be used for new indications after marketing (such as propranolol for hypertension). In this paper we seek to identify, as a basis for formulating policy, the central issues pertinent to a national system for the evaluation of drugs for both safety and efficacy after marketing. Surveillance for Safety N E E D FOR S U R V E I L L A N C E
When a decision to permit the marketing of a new drug is made, there are several reasons why the F D A has only incomplete knowledge of its side-effects. Among them are the following. First, phase 3 studies are necessarily of limited size (usually 700 to 3000 subjects in total). Adverse reactions that are uncommon (say, less than 1 per 1000 drug users) may not be detected. Second, many types of patients who will ultimately use the drug are excluded from phase 3 studies (such as persons in certain age groups, pregnant women, patients with diseases other than the one being studied, and patients concomitantly using other medications). This does not permit identification of side-effects that may occur only in certain subgroups of patients or effects which may arise because one drug modifies the action of another. Third, the duration of phase 3 studies tends to be quite limited, usually to 1 to 2 years. This does not permit identification of toxicity that manifests only after long ©1979 American College of Physicians
257
periods of use (such as chloroquine retinopathy [5]) or of side-effects appearing long after first exposure (such as adenocarcinoma of the vagina due to diethylstilbestrol [6]). From these considerations it follows that reassurance about a drug's safety requires long-term surveillance after marketing. Certain types of surveillance have already been in existence for some time (7), but the continuing discovery of important and previously unknown side-effects of drugs that have been available for many years (such as adenocarcinoma of the endometrium due to conjugated estrogens [8]) shows that the surveillance strategies so far implemented can fall short of providing the needed information with minimal delay. It is thus necessary to consider alternative or supplementary strategies that can serve as an adequate national system of drug surveillance. Such a system needs to cover effects of drugs in the fetus and other special populations. It should also be able to document effects of long-term and short-term drug use and effects of varying degrees of severity, and it should be able to identify effects that may only appear after long delays. To these needs a further requirement should be added: Even when an adverse effect is known to occur (whether from premarketing or postmarketing observations) only a proper quantitative estimate of its occurrence can form an adequate basis for therapeutic and regulatory decisions. A surveillance system should provide for such estimation. Furthermore, in carrying out this quantitative role, the system must be able to evaluate possible modification of adverse drug effects by factors such as various patient characteristics (such as age), concomitant drugs, and other factors (for example, the risk of myocardial infarction in women who use oral contraceptives seems to be augmented by cigarette smoking [9]). Taken together, the requirements for a comprehensive national surveillance system are quite formidable, and it is unlikely that they will ever be fulfilled completely. It seems generally to be agreed, however, that improvement of what already exists is possible and desirable. To quote Lasagna (4), "The conferees agreed that the question of whether or not post-marketing surveillance should be done is idle; that a relatively unstructured (open) system of post-marketing surveillance is now in existence; that post-marketing surveillance is an absolute necessity for assessing the benefit/risk ratio of a new drug; and that the proper object of discussion is the question of whether the present system should give way to some of the new (structured) proposals now under discussion, and to what extent."
OBJECTIVES OF S U R V E I L L A N C E
In view of the existing needs, the postmarketing surveillance of drugs for safety should serve three broad objectives: first, the discovery of side-effects unknown at the time of marketing, including those that may subsequently lead to new indications for drug use; second, confirmation of side-effects reported from other sources; and third, the quantitative evaluation of the risks of known 258
adverse effects, whether discovered before or after marketing. S U R V E I L L A N C E SYSTEM COMPONENTS
To meet these objectives, the F D A needs access to information on drug experience that is both timely and of good scientific quality. The attainment of this ideal requires a national system of drug surveillance. This has been recognized for a long time. In 1966, Finney (10) stated that " . . . the monitoring of adverse reactions must become part of the regular framework of health services . . . it is a response to a need to replace haphazard collection and interpretation of information by something more organized . . . " This view was supported in 1970 by the International Conference on Adverse Reaction Reporting Systems (11): "A major conclusion . . . was that the only protection . . . is in an effectively organized system for detecting adverse reactions . . . " Such a national system involves various components fulfilling different needs. Routine Data Systems: Effective surveillance requires systematic recording of drug experience. Two major types of routine data systems are currently in existence. First, there are nationwide programs that register putative adverse reactions; they rely on informal, ad hoc judgments of causality on the part of the reporters. Examples are the existing adverse reaction reporting system maintained by the F D A and the industry-based monitoring of drug experience. Such registries can be used to raise hypotheses of adverse effects. Such use involves the collation and evaluation of putative case reports sent directly to a registry or appearing in medical journals. Beyond the existing adverse reaction reporting schemes, other possibilities remain to be explored. Second, there are formal government-sponsored epidemiologic programs of drug surveillance, each concerned with a limited segment of drug experience (such as the Boston Collaborative Drug Surveillance Program [12], the Drug Epidemiology Unit [13], and the Pediatric Drug Surveillance Program [14]). These systems differ from the registries by using routine accumulation of primary data and formal epidemiologic analyses. In these analyses connections between drugs and side-effects are inferred in the aggregate, and not in individual patients. At the first level of analysis, the identified associations constitute no more than "signals" of possible causal connections. However, beyond simply detecting a potential cause-and-effect relation, the formal epidemiologic systems sometimes permit detailed scientific evaluation of an initial finding (that is, they can allow for the next step of ad hoc research seeking to verify the finding, explore it in detail, and evaluate it in quantitative terms). In any particular instance the ad hoc capability of a surveillance program depends on the nature of the possible connection and the types of data recorded. Capability for Ad Hoc Research: The evaluation of hypotheses, whatever their origin, would be enhanced by the maintenance of clinical-epidemiologic capabilities for the rapid and efficient conduct of various specific studies. Such projects may be prompted by case reports or articles, by associations discovered in the course of routine
February 1979 • Annals of Internal Medicine • Volume 90 • Number 2
Downloaded From: http://annals.org/ by a University of Otago User on 12/09/2016
surveillance, or by suspicions arising from some other source. These capabilities could be extensions of routine surveillance systems, supplementing their activities. The essential features of such "institutionalized capabilities" would be their rapid responsiveness to need, together with economy. This would be achieved by the maintenance of both the access to sources of the requisite new data, and the personnel and facilities to carry out the studies. Marshalling Extramural Resources: In instances of special need for ad hoc studies, effective use should be made of the various scientific resources that exist independently of the national system of surveillance: Certain research groups may have access to particularly useful data collected for other purposes, and they may be able to carry out the needed studies most expeditiously. For example, the most detailed study showing that oral contraceptives appear to cause benign liver tumors made use of an Army pathology registry (15). Effective use of such resources may call for national organization if they are to be mobilized speedily when needed—an aspect of an overall national system. Laboratory Research: The broad postmarketing experience with a drug may be complemented usefully by the maintenance of laboratory and other facilities to explore the more detailed nature, including mechanisms, of putative newly detected adverse drug effects. Conversely, laboratory discoveries may point to the need for ad hoc epidemiologic studies. RELATIONS OF COMPONENTS TO OBJECTIVES
In its broadest outline, a national system would be within the rubric of a national coordinating center having access to various research resources, available to be applied as appropriate. The roles of the components would vary according to whether the objective is one of discovery, verification, or quantitative evaluation. Discovery: All of the components referred to above can play a role in discovery. If an adverse event is acute and rare in the absence of drug use (such as sudden deafness), the discovery of its acute induction by a drug (for example, furosemide [16]) can be accomplished readily by clinical observation. If, on the other hand, the event is quite common (as is the case, for example, with many cancers), its discovery tends to depend on formal epidemiologic studies, whether carried out within the context of surveillance or on an ad hoc basis. When there is a long delay in the appearance of the adverse effect (such as adenocarcinoma of the uterus due to conjugated estrogens [8]), the need for an epidemiologic approach becomes even more compelling. Finally, laboratory research in animals may first suggest the possibility that a drug is causing some adverse effect. Verification of Reported Side-Effects: The choice of the appropriate strategy would depend on the nature of the problem. A hypothesis proposed elsewhere can be tested with existing surveillance data (17) or with ad hoc studies. Such a study is needed, for example, when case reports implicate drugs in the causation of very rare effects
(for example, benign liver tumors due to oral contraceptives [18]). Such effects are too uncommon to be represented in sufficient numbers in routine data systems. Quantitative Evaluation: Sometimes ad hoc research with existing surveillance-generated data bases can be used for quantitative evaluation. The alternatives are ad hoc research carried out using facilities sponsored by a national surveillance system or ad hoc research based on the marshalling of extramural resources. For quantitative evaluation there would not be a role for registries for adverse-reaction reports or for laboratory research concerned with mechanisms. S U R V E I L L A N C E OF N E W L Y M A R K E T E D D R U G S
A policy question of particular concern to the F D A is the organization of a system of surveillance to ensure that unrecognized risks associated with newly marketed drugs are detected as soon as possible after their release (7). In general, such surveillance would entail follow-up of a cohort of the earliest patients treated with the new drug. Appropriate nonexposed cohorts may also be needed for discovery and quantitation of effects. The efficiency of this approach depends on the frequency of the adverse events in those not exposed and on the magnitude of the effect caused by the drug. As yet, a routine system for the formation of such cohorts does not exist, although some ad hoc studies of new drugs have been carried out and have yielded worthwhile results (19). There remains the problem of potential serious adverse effects that are too rare or delayed to be detected in any cohort study of practicable size and duration. A case in point is the alleged connection between the use of cimetidine and bone marrow depression (20). Surveillance for such side-effects would entail monitoring various rare illnesses for changes in incidence, and the use of case-control studies. For both, the central system element would be a registry based upon a very large population. The Commission on Professional and Hospital Activities for example, records approximately 40% of United States hospital diagnoses, and already collaborates with the Center for Disease Control in monitoring congenital malformations (21). Efficacy Evaluation
In the past, the predominant social concern has been with potential side-effects of marketed drugs. However, there are policy issues related to efficacy that also merit consideration. N E E D FOR E V A L U A T I O N
Phase 3 studies of efficacy frequently require supplementation after marketing. The premarketing studies do not document long-term efficacy, efficacy in the context of everyday clinical practice, or efficacy in terms of any new indication for use suggested after marketing. Long-Term Efficacy: The deficiency of the phase 3 studies in the assessment of long-term efficacy derives from their short duration: They seldom last more than 1 or 2 years. Thus, while they can show, for example, that hypoglycemic agents lower blood glucose levels, they do Sloneetal. • Drug Evaluation After Marketing
Downloaded From: http://annals.org/ by a University of Otago User on 12/09/2016
259
not show that such drugs reduce the risk of diabetic complications (22). Efficacy in Clinical Practice: The phase 3 studies tend not to be representative of clinical practice in that they involve rigid adherence to treatment regimens, restricted eligibility for therapy, and other limitations. After marketing, persons who would have been excluded from the premarketing studies are likely to receive the drug, and it may be used in ways quite different from those in a clinical trial (such as dosage, adherence, concomitant drugs). New Indications: As already mentioned, a side-effect (whether identified in premarketing studies or only after marketing) can become a new indication. Alternatively, it may simply be realized, after marketing, that a drug has a previously unrecognized indication (such as phenytoin for the treatment of arrhythmias [23]). Whatever the basis for the suggested new indication, a test of its validity may require that the proposed new indication be subjected to rigorous evaluation analogous to that required in phase 3 studies. In short, the generalizability and relevance of phase 3 studies to the setting in which drugs ultimately come to be used can be severely limited.
E L E M E N T S R E Q U I R E D FOR T H E E V A L U A T I O N OF EFFICACY AFTER MARKETING
The nonexperimental methods used in epidemiologic research are of little value in efficacy research, and clinical trials are usually needed. The reason is that patients generally are selected for therapy with a given drug in such a way that they may differ from those not receiving the drug in their expected clinical course. Thus the indication for drug use is commonly a confounding factor in efficacy research. Often, this is a situation in which the problem of bias due to confounding cannot be handled using nonexperimental methods: usually it is not feasible to record all reasons for prescribing a particular drug. This issue of confounding is sufficiently important and subtle to merit consideration of a detailed example. Suppose that in a randomized double-blind trial propranolol reduces the risk of sudden death among survivors of myocardial infarction. A nonexperimental epidemiologic study might well produce a contrary (and invalid) result because of confounding by the indication for therapy: It is likely that the drug will be prescribed to those thought to have the highest risk of sudden death (inferred, for example, from persistent ventricular arrhythmias). Consequently, the propranolol recipients might tend to show a higher rate of sudden death than the nonrecipients, even though, in fact, the drug reduces the risk. Moreover, the particulars of arrhythmia and other factors leading to the use of this drug (and to adherence to therapy), in any given patient, tend to be too complex and subtle to be fully recorded. This makes detailed allowance for confounding by indication infeasible in the analysis of nonexperimental data. Because of uncontrollable confounding by indication in many, if not most, instances, nonexperimental epidemiologic studies will not suffice. Usually, there will be a need 260
for conducting randomized trials. In these trials, due consideration should be given to attaining some resemblance to the way the drug is used in actual practice. ORGANIZATIONAL FRAMEWORK
As with surveillance for side-effects it may be necessary to implement efficacy evaluation in the postmarketing phase as a nationally coordinated effort. Long-Term Efficacy: That there is need for randomized trials to assess long-term efficacy is borne out by experience. On the one hand, such trials have served to dispel misgivings about a drug's value. Outstanding examples are the Veterans Administration studies showing that therapy for hypertension reduces the risk of stroke (24, 25)). On the other hand, some trials have indicated the need for caution (for example, the finding that high doses of conjugated estrogens given to survivors of myocardial infarction increase, rather than diminish, the risk of subsequent infarction [26]). Although there is no great difficulty in showing that a drug reduces, say, cholesterol levels, the type of clinical trial needed to show that it reduces the risk of ischemic heart disease is technically complex. For such a trial to be practical at all requires a large organization having a coordinating center and interactions with collaborating institutions (27). From the policy perspective of having a national system, consideration should be given to the extent to which any center should be maintained on an institutionalized basis; and there may be a need for more than one center, with each specializing in different types of efficacy studies. Efficacy in Everyday Practice: It is something of a paradox that the very rigor of the orthodox clinical trial limits the generalizability of its findings; and the methods needed to achieve generalizability by carrying out clinical trials in the ordinary clinical setting have hardly been explored. Some tentative explorations of short-term efficacy, in a routine hospital ward setting, have been made with encouraging results (28, 29). This approach merits further exploration. With regard to the evaluation of the long-term efficacy of any drug, as used in everyday practice, no research has been done. Possibly, existing health maintenance organizations might be used to assign drugs on a randomized, double-blind basis to evaluate their long-term efficacy in practice. Evaluation for New Indications: As already pointed out, suggested new indications generally require evaluation in a clinical trial. Whether a new indication should be assessed by a research group responsible for the postmarketing evaluation of drug efficacy or by the manufacturer would to some extent depend on the nature of the indication itself, as well as the nature of the drug. ACKNOWLEDGMENTS: Many of the ideas expressed by the authors were developed in the course of their collaboration, as consultants to IMS America, Ltd., in preparing a Food and Drug Administration-sponsored report (7) on the problems of postmarketing surveillance. The authors are solely responsible, however, for the ideas expressed here. • Requests for reprints should be addressed to Dennis Slone, M.D.; Drug Epidemiology Unit, 10 Moulton Street; Cambridge, MA 02138. Received 27 November
February 1979 • Annals of Internal Medicine • Volume 90 • Number 2
Downloaded From: http://annals.org/ by a University of Otago User on 12/09/2016
1978; accepted 28 November
1978.
References
15. BOURNEROOKS J, O R Y HW, ISHAK K G , STRAUSS LT, G R E E N S P A N JR,
1. Interim Report. Expansion of FDA's statutory authority in the postmarketing period for new drugs. Review Panel on New Drug Regulation (DPHEW), Rockville, Maryland, 1977. 2. Hazards of non-practolol beta-blockers (editorial). Br Med J 1:529-530, 1977
TYLER CW: The association between oral contraception and hepato-cellular adenoma—a preliminary report. Int J Gynecol Obstet 15:143-149, 1977 16. COOPERMAN LB, RUBIN IL: Toxicity of ethacrynic acid and furosemide. Am Heart J 85:831-834, 1973 17. BOSTON COLLABORATIVE D R U G S U R V E I L L A N C E P R O G R A M M E : Oral
contraceptives and venous thromboembolic disease, surgically confirmed gallbladder disease, and breast tumours. Lancet 1:1399-1404, 1973
3. ZACHARIAS FJ, C O W E N KJ, PRESST J, VICKERS J, W A L L BG: Propra-
4. 5. 6. 7.
8.
nolol in hypertension: a study of long-term therapy, 1964-1970. Am Heart 7 83:755-761, 1972 LASAGNA L: Post-Marketing Surveillance of Drugs. Washington, D.C., Medicine in the Public Interest, Inc., 1976, p. 6 BERNSTEIN HN: Chloroquine ocular toxicity. Surv Ophthalmol 12:415447, 1967 DES Task force summary report. Part II: DES Daughters (DHEW publication no. (NIH) 79-1688), Bethesda, Maryland, 1978 IMS AMERICA LTD.: Health Care Services Research Group: Final Report—task A. An Experiment in Early Post-Marketing Surveillance of Drugs. ( D P H E W / P H S / F D A . Rockville, Maryland), 1978 WEISS NS: Interpretation of the association between exogenous estrogens and endometrial cancer, in Epidemiological Evaluation of Drugs,
between benign hepatomas and oral contraceptives. Lancet 2:926-929, 1973 19. Azaribine (triazure) recalled. FDA Drug Bull 6:34, 1976 20.
D E GALOCSY C, VAN Y P E R S E L E D E STRIHOU C: Pancytopenia with
edited by C O L O M B O F, SHAPIRO S, SLONE D, T O G N O N I G. Littleton,
cimetidine (letter). Ann Intern Med 90:214, 1979 21. The birth defects monitoring program. Am Coll Obstet Gynecol Newslet, February 1974 22. The University Group Diabetes Program. A study of the effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes. II. Mortality results. Diabetes 19: (Suppl 2), 787-830, 1970 23. HARRIS AS, KOKERNOT RH: Effects of diphenylhydantoin sodium and phenobarbital sodium upon ectopic ventricular tachycardia in acute myocardial infarction. Am J Physiol 163:505-516, 1950
Massachusetts, Publishing Sciences Group, Inc., 1977, pp. 161-167
24. V E T E R A N S A D M I N I S T R A T I O N C O O P E R A T I V E STUDY G R O U P ON A N -
9. M A N N JI, D O L L R, T H O R O G O O D M, VESSEY MP, W A T E R S WE: Risk
factors for myocardial infarction in young women. Br J Prev Soc Med 30:94-100, 1976 10. FINNEY DJ: Monitoring adverse reactions to drugs—its logic and its weaknesses, in Proceedings of the European Society for the Study of Drug Toxicity. Amsterdam, Exerpta Medica Foundation, 1966, p. 198 11. Report of the International Conference on Adverse Reaction Reporting Systems. Washington, D.C., National Academy of Sciences, 1971, p. 5 12.
18. B A U M JK, H O L T Z F, BOOKSTEIN JJ, K L E I N EW: Possible association
JlCK H, MlETTINEN O S , SHAPIRO S, LEWIS G P , SlSKIND V, SLONE D :
Comprehensive drug surveillance. JAMA
213:1455-1460, 1970
TIHYPERTENSIVE AGENTS: Effects of treatment on morbidity in hypertension: results in patients with diastolic blood pressure averaging 115 through 129 mm Hg. JAMA 202:1028-1034, 1967 25. V E T E R A N S A D M I N I S T R A T I O N C O O P E R A T I V E STUDY G R O U P ON A N -
TIHYPERTENSIVE AGENTS: Effects of treatment on morbidity in hypertension. II. Results in patients with diastolic blood pressure averaging 90 through 114 mm Hg. JAMA 213:1143-1152, 1970 26. T H E CORONARY D R U G P R O J E C T R E S E A R C H G R O U P : The Coronary
Drug Project. Findings leading to discontinuation of the 2.5-mg/day estrogen group. JAMA 226:652-657, 1973
13. SLONE D, SHAPIRO S, M I E T T I N E N OS: Case-control surveillance of seri-
27. T H E CORONARY D R U G P R O J E C T R E S E A R C H G R O U P : The Coronary
ous illnesses attributable to ambulatory drug use. See reference 8, pp. 5970
Drug Project. Design, methods, and baseline results. Circulation 47: (Suppl 1), 1973
14. M I T C H E L L AA, G O L D M A N P, SHAPIRO S, SISKIND V, SLONE D: Inten-
28. JICK H, SLONE D, SHAPIRO S, L E W I S G P , SISKIND V: A new method
sive drug surveillance in a pediatric hospital, in Clinical Pharmacy and Clinical Pharmacology, edited by GOUVEIA WA, TOGNONI A, V A N D E R KLEIJN E. Amsterdam, Elsevier/North Holland Biomedical Press, 1976, pp. 303-310
for assessing the clinical effects of oral analgesic drugs. Clin Pharmacol Ther 12:456-463, 1971 29. JICK H, SLONE D, SHAPIRO S, L E W I S G P : Clinical effects of hypnotics.
I. A controlled trial. JAMA
Slone etal.
Downloaded From: http://annals.org/ by a University of Otago User on 12/09/2016
209:2013-2015, 1969
• Drug Evaluation After Marketing
261
