Review
CLINICAL TRIAL DESIGN IN DERMATOLOGY: EXPERIMENTAL DESIGN. PART 1 ALFRED M. ALLEN, M.D., M.P.H.
from the Department of Dermatology Research, Letterman Army lnstrtute of Research, San Francisco, California
involved; therefore, this subject will be addressed separately in this series of papers on clinical trial design in dermatology. Why should clinical investigators be concerned with experimental design? Why not continue on with simple comparative trials? Or why not make ad hoc modifications to a basic design based on what seem reasonable clinical considerations? The answers to these questions reside in the fact that clinical trials have greatest meaning when the data can be subjected to rigorous logical and statistical analyses. Suppose, for example, it i s impossible to tell at the end of a trial whether the difference in the clinical outcomes of the 2 treatment groups i s due to the treatments or to other factors, such as severity of disease. The creation of such a dilemma is known in experimental design jargon a5 confounding; it can be avoided by choosing a suitable design. If there are no faults in the logical structure of a trial, differences in outcomes can be attributed to differences in efficacy between treatments, provided that statistical significance tests show the results could not easily have arisen by chance alone. Problems often lie in the statistical rather than in the logical structure of a trial. This involves the concept of statistical efficiency, which can be thought of in terms of the relative numbers of patients required to demonstrate (with statistical significance) a given difference between treatments.' How efficient a trial is in collecting information from a given
A Clinical trial i s a therapeutic experiment. Like all experiments, clinical trials have an organizational structure known technically as an experimental design. By the design ofan experiment is meant: ( 1 ) the set of treatments selected for comparison, (2) the specification of the units (patients or lesions) to which the treatmentsare to be applied, (3) the rules by which the treatments are to be allocated to experimental units, and (4) the specification of the measurements to be made on each unit? In clinical trials, the set of treatments selected for comparison i s usually decided by what medications or other therapeutic modalities (e.g., surgical procedures) are available for testing. The specification of the units to which the treatments are to be applied, and the rules by which the treatments are to be allocated to experimental units, are the elements usually stressed in textbooks of experimental design,'-i and are the subject of this paper. The specification of the measurements to be made on each unit can be especially difficult when dermatologic conditions are Address for reprints: Alfred M. Allen, M.D., Chief, Health and Environvent Activity, Fort Ord, CA 93941. 001 f-9059-78-01
OC-0042-0105
@ International Society of Tropical Dermatology
42
No. 1
EXPERIMENTAL DESIGN
number of patients frequently depends on quite simple but nonetheless profoundly important variations in experimental design. For example, an investigator using an inefficient design might require 150 patients to demonstrate a given difference between two treatments, whereas another investigator using a more efficient design might require only 50 patients to show the same difference. This threefold difference in efficiency could come about by introducing such design variations as pairing, blocking, and using the patient as his own control. The optimum choice of a trial design depends on the nature of the disease process, the characteristics of the patients, the effects of treatment, the kinds of measurements taken, and, under certain circumstances, ethical considerations. Experimental design may become exceedingly complex; however, we will consider only the simpler designs since these are the most useful clinically. The following sections illustrate 6 basic experimental designs.
Simple Comparisons This design i s the simplest and the most familiar to clinicians. In it, the subjects are divided into 2 groups: one receives treatment A and the other receives treatment B. The results are analyzed to see how treatment A compares with treatment B. The term treatment i s not restricted to its strict medical sense. For example, a placebo, a vehicle, or even no treatment at all is spoken of as allocation of a “treatment.” Even the simplest trial must be designed to answer a question or a group of related questions. In the example, the question is: H o w does treatment A compare with treatment B ? This question is deceptively simple; it actually contains the essence o i several questions. With properly collected data, the question could become: How does treatment A compare with treatment B in (1) efficacy, (2) incidence of adverse effects, and (3) convenience and acceptability? Suppose that in a trial of 2 steroid creams the questions of side effects and acceptability
Allen
43
are secondary to effectiveness. When efficacy is the primary focus, what must be considered to make the trial as informative as possible? In a simple comparison, the investigator will allocate Cream A to one group and Cream B to the other. He may, because of administrative convenience or implicit faith in his impartiality, make a non-random allocation of the patients to treatment groups; that is, he does not toss a coin or use a random number table or some other randomizing device, such as Social Security numbers. (See standard texts for techniques of r a n d ~ m i z a t i o n . ) ~ . ~ Randomization is essential to prevent bias. Subconscious bias may, for example, lead the investigator to allocate the patients with less severe cases to the new treatment. In that event, it is likely to be impossible to determine whether a difference in outcomes is due to a difference in efficacy. For maximum efficiency, it is necessary to divide the patients into equal-sized treatment groups. Deviation from equality in numbers reduces efficiency: the greater the deviation, the greater the loss of efficiency. With inefficient designs, more patients and more work are required to gather a given amount of information about the differences between treatments. Creating treatment groups of unequal size appears to be rooted in the understandable desire to gather more data on a new form of treatment than on an established, well-known form. This kind of thinking seems reasonable but can be statistically costly, especially when the ratio exceeds 2 : 1. Sometimes it may be desirable to keep one group as small as possible for ethical reasons. In these instances, one of the recently developed design variants i n which devices such as the “play the winner rule” are used may meet the investigator’s needs?
Paired Comparisons Paired comparisons allow more rigorous evaluations than simple comparisons. In any therapeutic study, there are always extraneous sources of variation which weaken the
INTERNATIONAL JOURNAL OF DERMATOLOGY
44
JanuaryIFebruary 1978
Vol. 17
Table 1. Unpaired and Paired Comparative Trials o f 2 Treatments for Pyoderma Erythromycin Type of pyoderma A. Unpaired Comparison Vesicular impetigo Ecthyma Furuncle
Days to heal
6 10 14
-
New Treatment Type of pyoderma Days to heal
lmpetiginized dermatitis Cellulitis Felon
Total Total 30 Fallacious conclusion: The new treatment is no better than erythromycin for pyoderma. B. Paired Comparison Vesicular impetigo Ecthyma Furuncle Total
6
10 14
30
Vesicular impetigo Ecthyma Furuncle
6 10 14
30
3 7
10
Total 20 Valid conclusion: The new treatment cuts healing time of pyoderma by approximately a third as compared with erythromycin.
comparison by influencing the effects of one treatment or the other. The difference between treatments may be sharpened by reducing the sources of extraneous variation, in this case by pairing. An example will illustrate both the principle and the method. Suppose that the aim of a trial is to find out how a new form of systemic treatment for bacterial pyoderma compares with a standard treatment, oral erythromycin. The patients vary as to specific type of infectionimpetigo, ecthyma, furuncle, and so on. Two possibilities for allocating treatments to patients i mmediately present themselves. In the simplest design, the patients are randomly assigned to one of the 2 treatments, and a groupto-group comparison is made. Alternatively, the patients may be paired according to such factors as the severity or type of disease. Then one member of each pair i s randomly assigned to one treatment and the partner to the other. The special advantage of pairing i s that differences between treatment groups can be more safely ascribed to the effects of treatment than to interfering effects from differences in such factors as disease type or severity. Paired comparisons also tend to be more efficient statistically than unpaired comparisons.L Table 1 shows a hypothetical example of the difference between paired and unpaired
comparisons in evaluating the treatment of pyoderma. In actual practice, many more patients would be enrolled so as to increase the chances of demonstrating a difference. With small sample sizes, the principle can be illustrated more forcefully. There i s another type of paired comparison which i s especially appropriate i n dermatologic studies. Rather than sorting the patients out into pairs, it is often possible to use symmetrical matching, otherwise known as the “method of simultaneous symmetrical paired c ~ m p a r i s o n s . ” ~In this case the 2 treatments are applied to opposite sides of the body. This technique is extremely usef.ul when the lesions are symmetrically distributed and of equal severity on both sides of the body. Topical steroids are often compared in this fashion in psoriasis or atopic dermatitis. This design miminizes extraneous sources of variation. However, translocation of potent drug from one site to another limits the application of this method.*
Crossover Designs In some situations it is possible to use the patient as his own control. This has 2 distinct advantages over the simple comparative design. First, it effectively doubles (and in some
No. 1
EXPERIMENTAL DESIGN
45
. Allen
Table 2. A Typical Block Design
Patient’s No. in each block 1 2 3 4 5 6
Experimental Design in Clinical Trials Male 20-39 yrs 40-59 yrs 20-39 yrs Block A Block B Block C T P T P P T
T T P P P T
P T T P
P T
Female 40-59 yrs Block D T P P T T P
T = treatment; P = placebo. Treatments assigned at random within each block.
cases triples or quadruples) the number of experimental subjects available for study. Second, efficiency i s increased because the comparison is paired. In the 2-period crossover design, one group receives cream A first and cream B second. The other group receives the drugs in the reverse order. The design looks like this:
First treatment Second treatment
Group 1
Group 2
A B
B A
Both groups receive both treatments. The only difference between the groups is the order in which they receive the treatments. In this design the number of patients required i s half ascompared with the simple comparative design. Crossover designs are limited to the study of disease processes in which the treatment i s short-acting and the patient’s condition i s expected to revert to its original state once treatment effects are gone. Conditions which are permanently altered by the course of treatment or the passage of time are generally not suitable for crossover studies. Attention must be paid to so-called carryover effects and rest periods. (The latter are also referred to as wash-out periods.) If the first treatment has a prolonged action, its effect will still be present when the second one i s given, thereby introducing a bias. The length of the wash-out period will depend on one’s
knowledge of the disease and the drugs being given. Sufficiently long wash-out periods between ending one treatment and starting another allow the effects of the first treatment to ”wash-out,” thus avoiding bias.
Randomized Blocks The randomized block design has been described by one authority as “the most valuable of all experimental designs, the most frequently used, and, except for the completely randomized, the simplest in construckion and statistical analysis.”l It i s simply an extension ofthe paired comparison design to instances in which there are more than a pair of experimental units with like characteristics. In a paired comparison the pair may consist of 2 patients who are matched o n the basis of age and sex; in a block design the block will then consist of any larger number of patients who arealike in age and sex. A typical blockdesign i s shown in Table 2. If the outcome of treatment is likely to be influenced by age and sex, then the randomized block design produces more and better information than a completely randomized design for several reasons. First, treatment comparisons within each block are made on patients who are essentially alike. This helps insure that differences between treatment groups are due to the effects of treatment and not to some extraneous influence, such as age or sex. Second, each treatment i s administered equally as often within
33
0
13 40
_ -
0
28 40 40
-
_ _ _
16
1
2
40
6
5
0
-
6
3
5
70
0 6
-
30
50
6 20 65
7
14
86
13
12 14
20 30
50 9
18
20
1 %
26 40
lo 40 75 30 40
-
4
Severe
Total
-
8
-
-
25
-
-
_
4
10
13
9 10
20
24
8
-
20 16 Mild Moderate
-
No. Cases No. Cases Type of Cases
Treatment Cured No. -%
-
No.
-
65
40
65
90
Placebo Cured No. %
No. Cases Treatment Cured No. % %
o m am *0
Placebo Cured No. %
No. Cases
Treatment Cured
No. Cases
Placebo Cured No. %
No. Cases
Randomized block design
Trial 2 Simple comparative trial: stratification of results Trial 1 Simple comparative trial: results not stratified
Table 3.
Outcomes of 3 Clinical Trials Comparing a New Treatment for Warts to a Placebo
Trial 3
INTERNATIONAL JOURNAL OF DERMATOLOGY
January/February 1978
Vol. 17
each block. Such equality in numbers makes for maximum efficiency in design. Third, each of the age-sex groups is equally represented. Since each age-sex group (block) has been treated in the same way, it is possible to establish whether differences in therapeutic outcome are due to age or sex. Too often, the results of a clinical trial in which a simple design is used do not reveal to what extent the differences are due to the treatments themsdves or to extraneous sources of variation. Investigators who take such factors as age and sex into account by post hoc blocking usually find that the treatments being compared are not equally represented in each block. In fact, it is not uncommon to find that nearly all of the patients in one block (e.g., males aged 20-39) have received only 1 of the 2 drugs. In this instance, of course, it is impossible to say what the results would have been with the other treat ment. To illustrate further the differences between randomized block designs and the simple comparativedesign, let us compare the results obtained in 3 hypothetical clinical trials in which a new treatment for verruca vulgaris is being compared with a placebo (Table 3). It i s impossible to say whether the more favorable outcome of the treated group in Trial 1 is due to the new treatment or due to the fact that more of the patients with mild cases of warts were allocated to the treated group than to the control group. In Trial 2, this defect in design is partially remedied by stratification in analysis. Note, however, that so few patients with severe warts were allocated to the placebo group that it i s not possible to make a meaningful comparison. The randomized block design in Trial 3 assured that neither of these uninformative and irremediable outcomes mentioned would occur. The small amount of extra effort required to plan and implement the randomized block design was rewarded with a trial that yielded the maximum amount of information from the number of patients available for study.
No. 1
EXPERIMENTAL DESIGN
Occasions arise when it is desirable to have more than one block system to control for extraneous sources of variation. A handy design in many. such instances is the Latin square, which derives its name from an old mathematical puzzle. In the Latin square design, the effects of one extraneous source of variation are cleverly balanced against those from another extraneous source. This enables both extraneous sources of variation to be eliminated in analyzing the differences between treatments. A 4 x 4 Latin square looks like this:
. Allen
47
In practice, a single Latin square i s never used as the whole experiment. Instead, the design i s replicated (repeated) a sufficient number of times to ensure an adequate sample size. There are 576 possible permutations of the 4 x 4 design, and vastly greater numbers of possibilities for 5 x 5 and 6 x 6 designs. Although the intricate nature of each design would seem to preclude the possibility of randomization, random allocation of patients or lesions to treatment can nonetheless be achieved by appropriate technique^.',^ incomplete Blocks
I 1 2 3
4
A B C D
II B
Ill
C D
D A B
A
C
IV D A B C
How can Latin squares be used in clinical trials? Suppose an investigator wishes to compare the efficacy of 4 different topical medications, A, B, C, and D, for eczema. There are 2 principal sources of variation which may influence the results of treatment-the age of the patient and the severity of the lesion. Each variable can be divided into 4 categories based on differences in age or lesion severity. If Roman numerals are assigned to the 4 age categories, and Arabic numerals to the 4 categories of severity, we then have a means of fitting these into the Latin square design. In the 4 x 4 design example, the Roman numerals corresponding to different ages are referred to as “column headings” and the Arabic numerals designating degrees of severity are known as “row headings.” The block systems indicated by the row and column headings are interwoven in such a way that each treatment occurs once and only once in each row and each column. Thus, a perfectly balanced design is achieved with a maximum of economy and precision. This allows the effects of treatment to be extracted from the trial free of interference from differences in the patients’ ages and the severity of their lesions.
In the previous section, the only kind of blocks discussed were complete blocks; that is to say, blocks containing all of the different treatments. Sometimes the block size is too small to accommodate all the treatments under study, yet it is still desirable to retain the advantages of a block design. In these instances it is possible to use a design known as an incomplete block. To illustrate, suppose that an investigator wishes to compare 3 different medications A, B, and C with regard to their ability to provide temporary relief from pruritus due to urticaria. Since a symptom i s involved, efficacy must be judged from the patient’s own word about his condition. Because there i s no absolute scale of measurementfor itching, it i s necessary for the patient to express a preference of one treatment over another. Due to the limited duration of hives, it is onlyfeasible in this instance to compare 2 treatments at a time in each patient. The treatments are given in sequence and are separated by rest periods of appropriate length, as in crossover studies. At the end of the sequence, the patient i s asked to state his preference for the treatment given first or the treatment given second. An extremely simple incomplete block design illustrates a means of testing the difference between treatments with a maximum of economy and precision. If there are, say, 150 patients who can be enrolled in the trial, each
INTERNATIONAL JOURNAL OF DERMATOLOGY
48
January/February 1978
Vol. 17
Table 4. An Incomplete Block Design Patient nos.
First treatment Second treatment
1-25
26-50
51-75
76-1 00
101-1 25
1 2 6 1 50
A B
B A
A C
C A
B C
C B
of the 3 possible pairs of treatments can be assigned to 50 subjects. The design can be balanced so as to allow for any tendency of the patients to prefer either the first or second treatment administered, regardless of its inherent antipruritic effects. Such a design i s shown in Table 4, with the 3 treatments to be tested arranged to form blocks consisting of the 6 pairs AB, BA, AC, CA, BC, and CB. It is important to realize that the numbers assigned to the patients do not represent the sequence in which they are assigned to a certain pair of medications. Assignment i s made at random, with the proviso that 25 patients be placed in each group. Hypothetical results which might have been obtained from the balanced incomplete block design just described show how straightforward and unambiguous i s the analysis and interpretation of this kind of design (Table 5). A slightly more complex example of a balanced incomplete block design can be used to illustrate the principle further. Suppose there is a pressing need to know which of 5 congeners of a corticosteroid applied topically is most effective in clearing a dermatosis that selectively affects both the elbows and the knees in each patient. Prior experience has shown that patients with this dermatosis tend to differ markedly from each other in their Table 5.
Comparison of 3 Antipruritic Agents for Relief of ltching from Hives ~
Subjects testing
~~~
~~~~~
Preferred drug A B C
No preference
response to steroid treatment; however, all 4 affected areas in each individual patient tend to respond in about the same fashion as one another. In this situation it i s reasonable to think of each patient as a block and to allocate a different steroid congener at random to each of the 4 affected sites (elbows and knees). Since knees might perceptably differ from elbows in response to treatment, a balanced design would have to take this into account. In Table 6, the five steroid congeners are designated as A, B, C, D, and E; this is a balanced incomplete block design which would be suitable for the circumstances just described. Note that each member of each pair of blocks (e.g., I and II) is the same as the other except that the order in which the treatments are applied to the elbows and knees i s the opposite of one another. This enables any possible differences in response to treatment between knees and elbows to be taken into account in the analysis and interpretation of the trial. . Assistance in choosing an appropriate incomplete block design can be obtained from any of several textbooks devoted :to experimental design.l, 3, lo
Factorial Designs The factorial design was the brainchild of the greatest statistical genius of the twentieth century, an Englishman named Sir Ronald Aylmer Fisher. Fisher conceived a type of experimental design in which the effects of several factors (e.g., treatments), each at different levels (e.g., concentrations or dosages) could be evaluated simultaneously. The adoption of factorial designs produced a quantum leap in the amount of information which could be extracted from agricultural field trials, where
No. 1
EXPERIMENTAL DESIGN Table 6.
.
49
Allen
lncomplete Block Design
~
Elbows
I
II
111
IV
v
VI
VII
Vlll
IX
x
B
D
A
D
A
D
A
C
A
C
C
E
C
E
B
E
B
E
B
D
D
B
D
A
D
A
C
A
C
A
E
C
E
C
E
B
E
B
D
B
Knees
these designs were first applied. Similar benefit has been derived from the use of factorial designs in every other field of experimentation. Factorial designs differ from other experimental designs in that several factors may be tested simultaneously with no loss of precision and with resultant economy of effort. A simple dermatologic example of a factorial design would be as follows. Suppose that 2 different kinds of treatments have been proposed for acne, one a specific modification of fat intake in the diet, and the other a daily capsule of vitamin A. We refer to these 2 treatments as factor D (diet) and factor V (vi‘tamin). In this example, each factor can be administered at either of 2 levels: present, which we designate by a capital D or V, and absent, which we designate by the lower case letters d or v. For the two treatment regimens, there are 4 possible combinations: DV Dv dv dv
If one member of each set of 4 patients is allocated at random to each of these 4 treatment combinations, and if the total group of patients consists of a multiple of 4, the effect of each treatment can be calculated separately. To show the effect of diet (factor D), for example, we can align the 4 possible treatment combinations so that simple subtraction yields the desired result: D V - dV = effect of D in the presence of V Dv - dv = effect of D in the absence of V
The most interesting and versatile aspects of factorial designs are not revealed by these
calculations of the main effects of factors and levels. The unique characteristic of factorial designs is that the effects of combiningvarious factors at different levels can also be assessed. The application of this feature to medicine i s that physicians often want to know whether 2 different treatments are synergistic, antagonistic, or independent in their effects on a patient’s disease. In experimental design terms, synergism is known as positive interaction and antagonism as negative interaction. In the simple 2 x 2 factorial design used as our example, the 2 factors would be said to interact if their combined effect was different from the simple addition of their separate effects. Interaction in more complex factorial designs can be detected and measured by means of a sophisticated statistical technique (developed by RA Fisher) known as the Analysis of Variance.” With an increase in the number of factors, levels, and other variables, factorial designs rapidly become more elaborate than the simple one used as our example. To illustrate, if 3 treatments are used, each at 3 levels (e.g., full strength, half strength, and none at all), the number of possible combinations of treatments and levels i s 3 3 = 27. In clinical applications, it i s likely that use of a smaller number of combinations would still produce a meaningful result. The advice of a statistician is essential in determining what an adequate number of combinations would be.
Comment “The first step in the controlled trial is to decide precisely what it sets out to ~ r o v e . ” ~
50
INTERNATIONAL JOURNAL OF DERMATOLOGY
This admonition by Sir Bradford Hill, the father of the modern controlled clinical trial, cannot be overemphasized. A trial whose design i s based on vaguely worded questions is likely to yield vague answers. For example, if the aim of a trial is no more than to determine whether hydrocortisone plus neomycin i s better than hydrocortisone alone for allergic eczema, this may be the only question which is answered. This sounds desirable until one begins to ask some pertinent related questions, such as: Do the results apply equally to men as to women? And to adults as well as children? Does severity of lesions influence the response to treatment?Do secondarily infected lesions respond differently from noninfected lesions?The best and often the only hope of answering such questions i s to specify them precisely ahead of time during the planning phase of a tri,al. The choice of an optimum experimental design for a therapeutic trial depends not only on specifying the questions precisely, but also on the nature of the disease process, the kinds of patients available for study, and other constraints. For example, it i s foolish to use a crossover design when the disease is such that it does not revert to its original state shortly after discontinuation of treatment. The creation of blocks ("blocking") is probably the single most useful and versatile experimental design option open to the clinical investigator. Even i n what otherwise would be simple comparative trials, blocking can ensure that there is an equal number of subjects in each treatment group and thereby assure maximum efficiency of comparisons. In this application, each consecutive group of, say, 6 patients admitted to the trial consists of 3 receiving treatment A and 3 receiving treatment B. Randomization i s practiced within each block of patients to prevent bias in allocating patients to treatment groups. Blocking is especially useful in cooperative multicentric trials in which each center contributes a variable number of patients to the total. It is often the case that the patients in one center differ from those in another in several
JanuaryIFebruary 1978
Vol. 17
pertinent respects, such as average age or average severity of lesions. Blocking ensures that the various treatments will be distributed equally at each center. In addition, it provides a mechanism for ensuring that the treatment groups are balanced in regard to age, sex, severity of lesions, and so forth. Blocking also allows the data from the cooperating centers to be pooled more safely than otherwise would be the case, and enormously simplifies the task of statistical analysis and interpretation. Finally, it provides a quick and reliable means of spotting and isolating incongruities which may explain why treatment results at one center differ from those at the others. No mention has yet been made of sequential designs, which ostensibly provide definitive clinical answers m the shortest possible time and with the fewest possible numbers of patientsi2 Proponents of these designs claim that the advantages reside in fewer exposures of patients to unexpectedly harmful or ineffective treatments, more rapid adoption of maximally effective treatments, and significant savings of investigators' time and lab0r.9.'~ These advantages may outweigh any disadvantages when life-threatening diseases and potentially lethal forms of treatment are involved; but since the vast majority of dermatologic diseases and treatment modalities do not fall into these categories, the disadvantages must be given more weight. The principal disadvantages of sequential designs in dermatology are (1) the difficulty in comprehending and communicating the assumptions required to construct an optimum design, and ( 2 ) the information generated is seldom adequate for clinical needs. The full argument against sequential designs has been given e1se~here.I~
Epilogue No review of trial designs would be complete without mention of so-called "doubleblind" trials. This term has unintended and perhaps unfortunate connotations, leading some authorities to attack it with a vigor that undermines the faith of the medical commun-
No. 1
EXPERIMENTAL DESIGN
ity in the strength of the underlying concept.15 The double-blind device is invaluable for preventing a physician’s or patient’s knowledge of the treatment administered from biasing the results of a trial. Obviously, stating that a trial was “double-blind” i s nothing more than a ludicrous deception if conditions are such that either the patient or the physician can see through the scheme. But it i s even more a b surd to blame double-blind studies for faults in trial design that should properly be classified and dealt with under such headings as “patient selection” and “sample size.” Thorough understanding of the elements of trial design, in conjunction with clinical perspicacity and meticulous execution of studies, will go far to prevent error and promote advances in medical knowledge. Acknowledgments: Margaret R. Wrensch, M.S., and Albert M. Kligman, M.D., Ph.D, criticized the manuscript and made numerous suggestions for improvement.
References 1. Finney, D. 1.: Experimental Design and Its Statistical Basis. Chicago, University of Chicago Press, 1955, pp. 3, 43, 44, 51, 58. 2. Clarke, G. M.: Statistics and Experimental Design. New York, American Elsevier, 1969. 3. Cox, D:R.: Planningof Experiments. New York, John Wiley & Sons, 1958. 4. Hill, A. B.: Principles of Medical Statistics. 8th edition. New York, Oxford University Press, 1967, pp. 251, 355-357. 5. Fleiss, J. L.: Statistical Methods for Rates and Proportions. New York, John Wiley & Sons, 1973, pp. 35-38.
. Allen
51
6. Weinstein, M. C.: Allocation of subjects in medical experiments. N. Engl. 1. Med. 291:1278, 1974. 7. Sulzberger, M. 6.: Evaluation of therapeutic agents on the human skin: method of simultaneous symmetrical paired comparisons. Clin. Pharmacol. Ther. 3:1, 1962. 8. Marples, R. R., Kligman, A. M.: Limitations of paired comparisons of topical drugs. Br. J. Dermatol. 88:61, 1973. 9. Armitage, P.: Statistical Methods in Medical Research. Oxford, Blackwell Scientific Publications, 1971, pp. 240, 241, 415-425. 10. Cochran, W. C., and Cox, G. M.: Experimental Designs. 2nd edition. New York, John Wiley & Sons, 1957. 11. Snedecor, G . W., and Cochran, W. C.: Statistical Methods. 6th edition. Ames, 10, Iowa State University Press, 1967, pp. 258-338. 12. Milne, J. A,: Clinical sequential trials in dermatology. Br. J. Dermatol. 82, (Suppl.) 6:99, 1970. 13. Truelove, S. C.: Therapeutic trials. In Edited by Witt Medical Surveys and Clinical Trials. Witts, L. J. : 2nd edition. New York, Oxford University Press, 1964, pp. 157-160. 14. Oldham, P. D.: Measurement in Medicine. Philadelphia, J. 6. Lippincott Co, 1968, pp. 169-170. 15. Kligman, A. M.: The double-blind dilemma. JAMA 225:1658. 1973.
General Reading Hill’s book (ref. 4) and Truelove’s chapter (ref. 13) are the best starting places for those who wish to learn more, since each contains an elementary exposition of the principles of experimental design as applied to clinical trials. Finney’s monograph (ref. 1) and Armitage‘s text (ref. 9) are somewhat more difficult, but are more comprehensive. The books by Cox (ref. 3) and Cochran and Cox (ref. 10) are primarily for the serious student of experimental design but may be useful for reference.
The appearance of the code at the bottom of the first page of an article in this journal indicates the copyright owner’s consent that copies of the article may be made for personal or internal use, or for personal or internal use of specific clients. This consent is given on the condition, however, that the copier pay the stated per-copy fee through the Copyright Clearance Center, Inc., P.O. Box 765, Schenectady, New York 1230 1 , for copying beyond that permitted by Sections 107 or 108 of the U S . Copyright Law. This consent does not extend to other kinds of copying, such as copying for general distribution, for advertising or promotional purposes, for creating new collective works, or for resale.
CLINICAL TRIAL DESIGN IN DERMATOLOGY: EXPERIMENTAL DESIGN. PART 1 ALFRED M. ALLEN, M.D., M.P.H.
from the Department of Dermatology Research, Letterman Army lnstrtute of Research, San Francisco, California
involved; therefore, this subject will be addressed separately in this series of papers on clinical trial design in dermatology. Why should clinical investigators be concerned with experimental design? Why not continue on with simple comparative trials? Or why not make ad hoc modifications to a basic design based on what seem reasonable clinical considerations? The answers to these questions reside in the fact that clinical trials have greatest meaning when the data can be subjected to rigorous logical and statistical analyses. Suppose, for example, it i s impossible to tell at the end of a trial whether the difference in the clinical outcomes of the 2 treatment groups i s due to the treatments or to other factors, such as severity of disease. The creation of such a dilemma is known in experimental design jargon a5 confounding; it can be avoided by choosing a suitable design. If there are no faults in the logical structure of a trial, differences in outcomes can be attributed to differences in efficacy between treatments, provided that statistical significance tests show the results could not easily have arisen by chance alone. Problems often lie in the statistical rather than in the logical structure of a trial. This involves the concept of statistical efficiency, which can be thought of in terms of the relative numbers of patients required to demonstrate (with statistical significance) a given difference between treatments.' How efficient a trial is in collecting information from a given
A Clinical trial i s a therapeutic experiment. Like all experiments, clinical trials have an organizational structure known technically as an experimental design. By the design ofan experiment is meant: ( 1 ) the set of treatments selected for comparison, (2) the specification of the units (patients or lesions) to which the treatmentsare to be applied, (3) the rules by which the treatments are to be allocated to experimental units, and (4) the specification of the measurements to be made on each unit? In clinical trials, the set of treatments selected for comparison i s usually decided by what medications or other therapeutic modalities (e.g., surgical procedures) are available for testing. The specification of the units to which the treatments are to be applied, and the rules by which the treatments are to be allocated to experimental units, are the elements usually stressed in textbooks of experimental design,'-i and are the subject of this paper. The specification of the measurements to be made on each unit can be especially difficult when dermatologic conditions are Address for reprints: Alfred M. Allen, M.D., Chief, Health and Environvent Activity, Fort Ord, CA 93941. 001 f-9059-78-01
OC-0042-0105
@ International Society of Tropical Dermatology
42
No. 1
EXPERIMENTAL DESIGN
number of patients frequently depends on quite simple but nonetheless profoundly important variations in experimental design. For example, an investigator using an inefficient design might require 150 patients to demonstrate a given difference between two treatments, whereas another investigator using a more efficient design might require only 50 patients to show the same difference. This threefold difference in efficiency could come about by introducing such design variations as pairing, blocking, and using the patient as his own control. The optimum choice of a trial design depends on the nature of the disease process, the characteristics of the patients, the effects of treatment, the kinds of measurements taken, and, under certain circumstances, ethical considerations. Experimental design may become exceedingly complex; however, we will consider only the simpler designs since these are the most useful clinically. The following sections illustrate 6 basic experimental designs.
Simple Comparisons This design i s the simplest and the most familiar to clinicians. In it, the subjects are divided into 2 groups: one receives treatment A and the other receives treatment B. The results are analyzed to see how treatment A compares with treatment B. The term treatment i s not restricted to its strict medical sense. For example, a placebo, a vehicle, or even no treatment at all is spoken of as allocation of a “treatment.” Even the simplest trial must be designed to answer a question or a group of related questions. In the example, the question is: H o w does treatment A compare with treatment B ? This question is deceptively simple; it actually contains the essence o i several questions. With properly collected data, the question could become: How does treatment A compare with treatment B in (1) efficacy, (2) incidence of adverse effects, and (3) convenience and acceptability? Suppose that in a trial of 2 steroid creams the questions of side effects and acceptability
Allen
43
are secondary to effectiveness. When efficacy is the primary focus, what must be considered to make the trial as informative as possible? In a simple comparison, the investigator will allocate Cream A to one group and Cream B to the other. He may, because of administrative convenience or implicit faith in his impartiality, make a non-random allocation of the patients to treatment groups; that is, he does not toss a coin or use a random number table or some other randomizing device, such as Social Security numbers. (See standard texts for techniques of r a n d ~ m i z a t i o n . ) ~ . ~ Randomization is essential to prevent bias. Subconscious bias may, for example, lead the investigator to allocate the patients with less severe cases to the new treatment. In that event, it is likely to be impossible to determine whether a difference in outcomes is due to a difference in efficacy. For maximum efficiency, it is necessary to divide the patients into equal-sized treatment groups. Deviation from equality in numbers reduces efficiency: the greater the deviation, the greater the loss of efficiency. With inefficient designs, more patients and more work are required to gather a given amount of information about the differences between treatments. Creating treatment groups of unequal size appears to be rooted in the understandable desire to gather more data on a new form of treatment than on an established, well-known form. This kind of thinking seems reasonable but can be statistically costly, especially when the ratio exceeds 2 : 1. Sometimes it may be desirable to keep one group as small as possible for ethical reasons. In these instances, one of the recently developed design variants i n which devices such as the “play the winner rule” are used may meet the investigator’s needs?
Paired Comparisons Paired comparisons allow more rigorous evaluations than simple comparisons. In any therapeutic study, there are always extraneous sources of variation which weaken the
INTERNATIONAL JOURNAL OF DERMATOLOGY
44
JanuaryIFebruary 1978
Vol. 17
Table 1. Unpaired and Paired Comparative Trials o f 2 Treatments for Pyoderma Erythromycin Type of pyoderma A. Unpaired Comparison Vesicular impetigo Ecthyma Furuncle
Days to heal
6 10 14
-
New Treatment Type of pyoderma Days to heal
lmpetiginized dermatitis Cellulitis Felon
Total Total 30 Fallacious conclusion: The new treatment is no better than erythromycin for pyoderma. B. Paired Comparison Vesicular impetigo Ecthyma Furuncle Total
6
10 14
30
Vesicular impetigo Ecthyma Furuncle
6 10 14
30
3 7
10
Total 20 Valid conclusion: The new treatment cuts healing time of pyoderma by approximately a third as compared with erythromycin.
comparison by influencing the effects of one treatment or the other. The difference between treatments may be sharpened by reducing the sources of extraneous variation, in this case by pairing. An example will illustrate both the principle and the method. Suppose that the aim of a trial is to find out how a new form of systemic treatment for bacterial pyoderma compares with a standard treatment, oral erythromycin. The patients vary as to specific type of infectionimpetigo, ecthyma, furuncle, and so on. Two possibilities for allocating treatments to patients i mmediately present themselves. In the simplest design, the patients are randomly assigned to one of the 2 treatments, and a groupto-group comparison is made. Alternatively, the patients may be paired according to such factors as the severity or type of disease. Then one member of each pair i s randomly assigned to one treatment and the partner to the other. The special advantage of pairing i s that differences between treatment groups can be more safely ascribed to the effects of treatment than to interfering effects from differences in such factors as disease type or severity. Paired comparisons also tend to be more efficient statistically than unpaired comparisons.L Table 1 shows a hypothetical example of the difference between paired and unpaired
comparisons in evaluating the treatment of pyoderma. In actual practice, many more patients would be enrolled so as to increase the chances of demonstrating a difference. With small sample sizes, the principle can be illustrated more forcefully. There i s another type of paired comparison which i s especially appropriate i n dermatologic studies. Rather than sorting the patients out into pairs, it is often possible to use symmetrical matching, otherwise known as the “method of simultaneous symmetrical paired c ~ m p a r i s o n s . ” ~In this case the 2 treatments are applied to opposite sides of the body. This technique is extremely usef.ul when the lesions are symmetrically distributed and of equal severity on both sides of the body. Topical steroids are often compared in this fashion in psoriasis or atopic dermatitis. This design miminizes extraneous sources of variation. However, translocation of potent drug from one site to another limits the application of this method.*
Crossover Designs In some situations it is possible to use the patient as his own control. This has 2 distinct advantages over the simple comparative design. First, it effectively doubles (and in some
No. 1
EXPERIMENTAL DESIGN
45
. Allen
Table 2. A Typical Block Design
Patient’s No. in each block 1 2 3 4 5 6
Experimental Design in Clinical Trials Male 20-39 yrs 40-59 yrs 20-39 yrs Block A Block B Block C T P T P P T
T T P P P T
P T T P
P T
Female 40-59 yrs Block D T P P T T P
T = treatment; P = placebo. Treatments assigned at random within each block.
cases triples or quadruples) the number of experimental subjects available for study. Second, efficiency i s increased because the comparison is paired. In the 2-period crossover design, one group receives cream A first and cream B second. The other group receives the drugs in the reverse order. The design looks like this:
First treatment Second treatment
Group 1
Group 2
A B
B A
Both groups receive both treatments. The only difference between the groups is the order in which they receive the treatments. In this design the number of patients required i s half ascompared with the simple comparative design. Crossover designs are limited to the study of disease processes in which the treatment i s short-acting and the patient’s condition i s expected to revert to its original state once treatment effects are gone. Conditions which are permanently altered by the course of treatment or the passage of time are generally not suitable for crossover studies. Attention must be paid to so-called carryover effects and rest periods. (The latter are also referred to as wash-out periods.) If the first treatment has a prolonged action, its effect will still be present when the second one i s given, thereby introducing a bias. The length of the wash-out period will depend on one’s
knowledge of the disease and the drugs being given. Sufficiently long wash-out periods between ending one treatment and starting another allow the effects of the first treatment to ”wash-out,” thus avoiding bias.
Randomized Blocks The randomized block design has been described by one authority as “the most valuable of all experimental designs, the most frequently used, and, except for the completely randomized, the simplest in construckion and statistical analysis.”l It i s simply an extension ofthe paired comparison design to instances in which there are more than a pair of experimental units with like characteristics. In a paired comparison the pair may consist of 2 patients who are matched o n the basis of age and sex; in a block design the block will then consist of any larger number of patients who arealike in age and sex. A typical blockdesign i s shown in Table 2. If the outcome of treatment is likely to be influenced by age and sex, then the randomized block design produces more and better information than a completely randomized design for several reasons. First, treatment comparisons within each block are made on patients who are essentially alike. This helps insure that differences between treatment groups are due to the effects of treatment and not to some extraneous influence, such as age or sex. Second, each treatment i s administered equally as often within
33
0
13 40
_ -
0
28 40 40
-
_ _ _
16
1
2
40
6
5
0
-
6
3
5
70
0 6
-
30
50
6 20 65
7
14
86
13
12 14
20 30
50 9
18
20
1 %
26 40
lo 40 75 30 40
-
4
Severe
Total
-
8
-
-
25
-
-
_
4
10
13
9 10
20
24
8
-
20 16 Mild Moderate
-
No. Cases No. Cases Type of Cases
Treatment Cured No. -%
-
No.
-
65
40
65
90
Placebo Cured No. %
No. Cases Treatment Cured No. % %
o m am *0
Placebo Cured No. %
No. Cases
Treatment Cured
No. Cases
Placebo Cured No. %
No. Cases
Randomized block design
Trial 2 Simple comparative trial: stratification of results Trial 1 Simple comparative trial: results not stratified
Table 3.
Outcomes of 3 Clinical Trials Comparing a New Treatment for Warts to a Placebo
Trial 3
INTERNATIONAL JOURNAL OF DERMATOLOGY
January/February 1978
Vol. 17
each block. Such equality in numbers makes for maximum efficiency in design. Third, each of the age-sex groups is equally represented. Since each age-sex group (block) has been treated in the same way, it is possible to establish whether differences in therapeutic outcome are due to age or sex. Too often, the results of a clinical trial in which a simple design is used do not reveal to what extent the differences are due to the treatments themsdves or to extraneous sources of variation. Investigators who take such factors as age and sex into account by post hoc blocking usually find that the treatments being compared are not equally represented in each block. In fact, it is not uncommon to find that nearly all of the patients in one block (e.g., males aged 20-39) have received only 1 of the 2 drugs. In this instance, of course, it is impossible to say what the results would have been with the other treat ment. To illustrate further the differences between randomized block designs and the simple comparativedesign, let us compare the results obtained in 3 hypothetical clinical trials in which a new treatment for verruca vulgaris is being compared with a placebo (Table 3). It i s impossible to say whether the more favorable outcome of the treated group in Trial 1 is due to the new treatment or due to the fact that more of the patients with mild cases of warts were allocated to the treated group than to the control group. In Trial 2, this defect in design is partially remedied by stratification in analysis. Note, however, that so few patients with severe warts were allocated to the placebo group that it i s not possible to make a meaningful comparison. The randomized block design in Trial 3 assured that neither of these uninformative and irremediable outcomes mentioned would occur. The small amount of extra effort required to plan and implement the randomized block design was rewarded with a trial that yielded the maximum amount of information from the number of patients available for study.
No. 1
EXPERIMENTAL DESIGN
Occasions arise when it is desirable to have more than one block system to control for extraneous sources of variation. A handy design in many. such instances is the Latin square, which derives its name from an old mathematical puzzle. In the Latin square design, the effects of one extraneous source of variation are cleverly balanced against those from another extraneous source. This enables both extraneous sources of variation to be eliminated in analyzing the differences between treatments. A 4 x 4 Latin square looks like this:
. Allen
47
In practice, a single Latin square i s never used as the whole experiment. Instead, the design i s replicated (repeated) a sufficient number of times to ensure an adequate sample size. There are 576 possible permutations of the 4 x 4 design, and vastly greater numbers of possibilities for 5 x 5 and 6 x 6 designs. Although the intricate nature of each design would seem to preclude the possibility of randomization, random allocation of patients or lesions to treatment can nonetheless be achieved by appropriate technique^.',^ incomplete Blocks
I 1 2 3
4
A B C D
II B
Ill
C D
D A B
A
C
IV D A B C
How can Latin squares be used in clinical trials? Suppose an investigator wishes to compare the efficacy of 4 different topical medications, A, B, C, and D, for eczema. There are 2 principal sources of variation which may influence the results of treatment-the age of the patient and the severity of the lesion. Each variable can be divided into 4 categories based on differences in age or lesion severity. If Roman numerals are assigned to the 4 age categories, and Arabic numerals to the 4 categories of severity, we then have a means of fitting these into the Latin square design. In the 4 x 4 design example, the Roman numerals corresponding to different ages are referred to as “column headings” and the Arabic numerals designating degrees of severity are known as “row headings.” The block systems indicated by the row and column headings are interwoven in such a way that each treatment occurs once and only once in each row and each column. Thus, a perfectly balanced design is achieved with a maximum of economy and precision. This allows the effects of treatment to be extracted from the trial free of interference from differences in the patients’ ages and the severity of their lesions.
In the previous section, the only kind of blocks discussed were complete blocks; that is to say, blocks containing all of the different treatments. Sometimes the block size is too small to accommodate all the treatments under study, yet it is still desirable to retain the advantages of a block design. In these instances it is possible to use a design known as an incomplete block. To illustrate, suppose that an investigator wishes to compare 3 different medications A, B, and C with regard to their ability to provide temporary relief from pruritus due to urticaria. Since a symptom i s involved, efficacy must be judged from the patient’s own word about his condition. Because there i s no absolute scale of measurementfor itching, it i s necessary for the patient to express a preference of one treatment over another. Due to the limited duration of hives, it is onlyfeasible in this instance to compare 2 treatments at a time in each patient. The treatments are given in sequence and are separated by rest periods of appropriate length, as in crossover studies. At the end of the sequence, the patient i s asked to state his preference for the treatment given first or the treatment given second. An extremely simple incomplete block design illustrates a means of testing the difference between treatments with a maximum of economy and precision. If there are, say, 150 patients who can be enrolled in the trial, each
INTERNATIONAL JOURNAL OF DERMATOLOGY
48
January/February 1978
Vol. 17
Table 4. An Incomplete Block Design Patient nos.
First treatment Second treatment
1-25
26-50
51-75
76-1 00
101-1 25
1 2 6 1 50
A B
B A
A C
C A
B C
C B
of the 3 possible pairs of treatments can be assigned to 50 subjects. The design can be balanced so as to allow for any tendency of the patients to prefer either the first or second treatment administered, regardless of its inherent antipruritic effects. Such a design i s shown in Table 4, with the 3 treatments to be tested arranged to form blocks consisting of the 6 pairs AB, BA, AC, CA, BC, and CB. It is important to realize that the numbers assigned to the patients do not represent the sequence in which they are assigned to a certain pair of medications. Assignment i s made at random, with the proviso that 25 patients be placed in each group. Hypothetical results which might have been obtained from the balanced incomplete block design just described show how straightforward and unambiguous i s the analysis and interpretation of this kind of design (Table 5). A slightly more complex example of a balanced incomplete block design can be used to illustrate the principle further. Suppose there is a pressing need to know which of 5 congeners of a corticosteroid applied topically is most effective in clearing a dermatosis that selectively affects both the elbows and the knees in each patient. Prior experience has shown that patients with this dermatosis tend to differ markedly from each other in their Table 5.
Comparison of 3 Antipruritic Agents for Relief of ltching from Hives ~
Subjects testing
~~~
~~~~~
Preferred drug A B C
No preference
response to steroid treatment; however, all 4 affected areas in each individual patient tend to respond in about the same fashion as one another. In this situation it i s reasonable to think of each patient as a block and to allocate a different steroid congener at random to each of the 4 affected sites (elbows and knees). Since knees might perceptably differ from elbows in response to treatment, a balanced design would have to take this into account. In Table 6, the five steroid congeners are designated as A, B, C, D, and E; this is a balanced incomplete block design which would be suitable for the circumstances just described. Note that each member of each pair of blocks (e.g., I and II) is the same as the other except that the order in which the treatments are applied to the elbows and knees i s the opposite of one another. This enables any possible differences in response to treatment between knees and elbows to be taken into account in the analysis and interpretation of the trial. . Assistance in choosing an appropriate incomplete block design can be obtained from any of several textbooks devoted :to experimental design.l, 3, lo
Factorial Designs The factorial design was the brainchild of the greatest statistical genius of the twentieth century, an Englishman named Sir Ronald Aylmer Fisher. Fisher conceived a type of experimental design in which the effects of several factors (e.g., treatments), each at different levels (e.g., concentrations or dosages) could be evaluated simultaneously. The adoption of factorial designs produced a quantum leap in the amount of information which could be extracted from agricultural field trials, where
No. 1
EXPERIMENTAL DESIGN Table 6.
.
49
Allen
lncomplete Block Design
~
Elbows
I
II
111
IV
v
VI
VII
Vlll
IX
x
B
D
A
D
A
D
A
C
A
C
C
E
C
E
B
E
B
E
B
D
D
B
D
A
D
A
C
A
C
A
E
C
E
C
E
B
E
B
D
B
Knees
these designs were first applied. Similar benefit has been derived from the use of factorial designs in every other field of experimentation. Factorial designs differ from other experimental designs in that several factors may be tested simultaneously with no loss of precision and with resultant economy of effort. A simple dermatologic example of a factorial design would be as follows. Suppose that 2 different kinds of treatments have been proposed for acne, one a specific modification of fat intake in the diet, and the other a daily capsule of vitamin A. We refer to these 2 treatments as factor D (diet) and factor V (vi‘tamin). In this example, each factor can be administered at either of 2 levels: present, which we designate by a capital D or V, and absent, which we designate by the lower case letters d or v. For the two treatment regimens, there are 4 possible combinations: DV Dv dv dv
If one member of each set of 4 patients is allocated at random to each of these 4 treatment combinations, and if the total group of patients consists of a multiple of 4, the effect of each treatment can be calculated separately. To show the effect of diet (factor D), for example, we can align the 4 possible treatment combinations so that simple subtraction yields the desired result: D V - dV = effect of D in the presence of V Dv - dv = effect of D in the absence of V
The most interesting and versatile aspects of factorial designs are not revealed by these
calculations of the main effects of factors and levels. The unique characteristic of factorial designs is that the effects of combiningvarious factors at different levels can also be assessed. The application of this feature to medicine i s that physicians often want to know whether 2 different treatments are synergistic, antagonistic, or independent in their effects on a patient’s disease. In experimental design terms, synergism is known as positive interaction and antagonism as negative interaction. In the simple 2 x 2 factorial design used as our example, the 2 factors would be said to interact if their combined effect was different from the simple addition of their separate effects. Interaction in more complex factorial designs can be detected and measured by means of a sophisticated statistical technique (developed by RA Fisher) known as the Analysis of Variance.” With an increase in the number of factors, levels, and other variables, factorial designs rapidly become more elaborate than the simple one used as our example. To illustrate, if 3 treatments are used, each at 3 levels (e.g., full strength, half strength, and none at all), the number of possible combinations of treatments and levels i s 3 3 = 27. In clinical applications, it i s likely that use of a smaller number of combinations would still produce a meaningful result. The advice of a statistician is essential in determining what an adequate number of combinations would be.
Comment “The first step in the controlled trial is to decide precisely what it sets out to ~ r o v e . ” ~
50
INTERNATIONAL JOURNAL OF DERMATOLOGY
This admonition by Sir Bradford Hill, the father of the modern controlled clinical trial, cannot be overemphasized. A trial whose design i s based on vaguely worded questions is likely to yield vague answers. For example, if the aim of a trial is no more than to determine whether hydrocortisone plus neomycin i s better than hydrocortisone alone for allergic eczema, this may be the only question which is answered. This sounds desirable until one begins to ask some pertinent related questions, such as: Do the results apply equally to men as to women? And to adults as well as children? Does severity of lesions influence the response to treatment?Do secondarily infected lesions respond differently from noninfected lesions?The best and often the only hope of answering such questions i s to specify them precisely ahead of time during the planning phase of a tri,al. The choice of an optimum experimental design for a therapeutic trial depends not only on specifying the questions precisely, but also on the nature of the disease process, the kinds of patients available for study, and other constraints. For example, it i s foolish to use a crossover design when the disease is such that it does not revert to its original state shortly after discontinuation of treatment. The creation of blocks ("blocking") is probably the single most useful and versatile experimental design option open to the clinical investigator. Even i n what otherwise would be simple comparative trials, blocking can ensure that there is an equal number of subjects in each treatment group and thereby assure maximum efficiency of comparisons. In this application, each consecutive group of, say, 6 patients admitted to the trial consists of 3 receiving treatment A and 3 receiving treatment B. Randomization i s practiced within each block of patients to prevent bias in allocating patients to treatment groups. Blocking is especially useful in cooperative multicentric trials in which each center contributes a variable number of patients to the total. It is often the case that the patients in one center differ from those in another in several
JanuaryIFebruary 1978
Vol. 17
pertinent respects, such as average age or average severity of lesions. Blocking ensures that the various treatments will be distributed equally at each center. In addition, it provides a mechanism for ensuring that the treatment groups are balanced in regard to age, sex, severity of lesions, and so forth. Blocking also allows the data from the cooperating centers to be pooled more safely than otherwise would be the case, and enormously simplifies the task of statistical analysis and interpretation. Finally, it provides a quick and reliable means of spotting and isolating incongruities which may explain why treatment results at one center differ from those at the others. No mention has yet been made of sequential designs, which ostensibly provide definitive clinical answers m the shortest possible time and with the fewest possible numbers of patientsi2 Proponents of these designs claim that the advantages reside in fewer exposures of patients to unexpectedly harmful or ineffective treatments, more rapid adoption of maximally effective treatments, and significant savings of investigators' time and lab0r.9.'~ These advantages may outweigh any disadvantages when life-threatening diseases and potentially lethal forms of treatment are involved; but since the vast majority of dermatologic diseases and treatment modalities do not fall into these categories, the disadvantages must be given more weight. The principal disadvantages of sequential designs in dermatology are (1) the difficulty in comprehending and communicating the assumptions required to construct an optimum design, and ( 2 ) the information generated is seldom adequate for clinical needs. The full argument against sequential designs has been given e1se~here.I~
Epilogue No review of trial designs would be complete without mention of so-called "doubleblind" trials. This term has unintended and perhaps unfortunate connotations, leading some authorities to attack it with a vigor that undermines the faith of the medical commun-
No. 1
EXPERIMENTAL DESIGN
ity in the strength of the underlying concept.15 The double-blind device is invaluable for preventing a physician’s or patient’s knowledge of the treatment administered from biasing the results of a trial. Obviously, stating that a trial was “double-blind” i s nothing more than a ludicrous deception if conditions are such that either the patient or the physician can see through the scheme. But it i s even more a b surd to blame double-blind studies for faults in trial design that should properly be classified and dealt with under such headings as “patient selection” and “sample size.” Thorough understanding of the elements of trial design, in conjunction with clinical perspicacity and meticulous execution of studies, will go far to prevent error and promote advances in medical knowledge. Acknowledgments: Margaret R. Wrensch, M.S., and Albert M. Kligman, M.D., Ph.D, criticized the manuscript and made numerous suggestions for improvement.
References 1. Finney, D. 1.: Experimental Design and Its Statistical Basis. Chicago, University of Chicago Press, 1955, pp. 3, 43, 44, 51, 58. 2. Clarke, G. M.: Statistics and Experimental Design. New York, American Elsevier, 1969. 3. Cox, D:R.: Planningof Experiments. New York, John Wiley & Sons, 1958. 4. Hill, A. B.: Principles of Medical Statistics. 8th edition. New York, Oxford University Press, 1967, pp. 251, 355-357. 5. Fleiss, J. L.: Statistical Methods for Rates and Proportions. New York, John Wiley & Sons, 1973, pp. 35-38.
. Allen
51
6. Weinstein, M. C.: Allocation of subjects in medical experiments. N. Engl. 1. Med. 291:1278, 1974. 7. Sulzberger, M. 6.: Evaluation of therapeutic agents on the human skin: method of simultaneous symmetrical paired comparisons. Clin. Pharmacol. Ther. 3:1, 1962. 8. Marples, R. R., Kligman, A. M.: Limitations of paired comparisons of topical drugs. Br. J. Dermatol. 88:61, 1973. 9. Armitage, P.: Statistical Methods in Medical Research. Oxford, Blackwell Scientific Publications, 1971, pp. 240, 241, 415-425. 10. Cochran, W. C., and Cox, G. M.: Experimental Designs. 2nd edition. New York, John Wiley & Sons, 1957. 11. Snedecor, G . W., and Cochran, W. C.: Statistical Methods. 6th edition. Ames, 10, Iowa State University Press, 1967, pp. 258-338. 12. Milne, J. A,: Clinical sequential trials in dermatology. Br. J. Dermatol. 82, (Suppl.) 6:99, 1970. 13. Truelove, S. C.: Therapeutic trials. In Edited by Witt Medical Surveys and Clinical Trials. Witts, L. J. : 2nd edition. New York, Oxford University Press, 1964, pp. 157-160. 14. Oldham, P. D.: Measurement in Medicine. Philadelphia, J. 6. Lippincott Co, 1968, pp. 169-170. 15. Kligman, A. M.: The double-blind dilemma. JAMA 225:1658. 1973.
General Reading Hill’s book (ref. 4) and Truelove’s chapter (ref. 13) are the best starting places for those who wish to learn more, since each contains an elementary exposition of the principles of experimental design as applied to clinical trials. Finney’s monograph (ref. 1) and Armitage‘s text (ref. 9) are somewhat more difficult, but are more comprehensive. The books by Cox (ref. 3) and Cochran and Cox (ref. 10) are primarily for the serious student of experimental design but may be useful for reference.
The appearance of the code at the bottom of the first page of an article in this journal indicates the copyright owner’s consent that copies of the article may be made for personal or internal use, or for personal or internal use of specific clients. This consent is given on the condition, however, that the copier pay the stated per-copy fee through the Copyright Clearance Center, Inc., P.O. Box 765, Schenectady, New York 1230 1 , for copying beyond that permitted by Sections 107 or 108 of the U S . Copyright Law. This consent does not extend to other kinds of copying, such as copying for general distribution, for advertising or promotional purposes, for creating new collective works, or for resale.
