LETTERS TO THE EDITOR Psychotherapy Research To the Editor: The review of child psychotherapy research by Barrnett et al. (1991) is an example of a growing set of review articles concluding that methodological errors preclude any clear-cut inferences from studies that have absorbed huge numbers of person hours, dollars, and pages of articles. The image of so much wasted effort is sad; clarifying what is necessary for methodologically sound outcome research is thus a very high-priority task. The 17 methodological features listed by Barrnett et al. are not equally important. Some compete with others. Some are not desirable for certain studies. An insistence that all of the features be present might discourage the conduct or publication of experimental studies. I would suggest a hierarchical arrangement of methodological features, giving most weight to internal validity (i.e., whether the design permits seeing if the experimental manipulation really made a difference), least weight to features meant to increase relative efficiency (i. e., the power per subject), and intermediate weight to several other features that space doesn't permit discussing. Matching or blocking, one of the 17 features listed, is meant to increase relative efficiency, not to insure internal validity. In studies with random assignment to experimental groups, failure to match groups on variables, such as age, sex, and IQ, should not discount the validity of the results obtained. As Hulley and Cummings (1988) put it, "An important fact, not widely appreciated, is that the effects of any maldistributions that do occur as a result of chance (and on the average, one in every 20 baseline variables will differ at p < 0.05) are automatically included in the statistical tests .... " In other words, the chance that age, sex, IQ, or any other variable, measured or unmeasured, could have been maldistributed enough and could have had significant effects on the outcome variable to account for the difference between groups is less than the p value obtained. This is part of the beauty of randomization: it relieves the investigator of the impossible task of proving that the experimental and control groups are equivalent on every variable relevant to outcome. Matching of subjects before randomization does allow more power, i.e., a greater chance of obtaining statistical significance for any given effect size. However, increased power is also obtainable by having a larger sample size, which is often easier when matching is not insisted upon. Matching pairs of subjects on three variables, with three levels per variable, results in only 1/27 of the sample that is expected to be available as a match for any other given subject, assuming the bestcase scenario of equal distribution of cases among the levels. Matching is logistically difficult for the experimenter. It also introduces complications in the handling of missing data points. Kerlinger (1973), after weighing the advantages and disadvantages, states that "It must be emphasized that deliberate matching is in general not a desirable procedure. " A second example concerns sample homogeneity. Although it is obviously necessary to exert some selection for inclusion in a study, I remember instances where strict insistence on too homogeneous a sample has worked against the acceptance of good research proposals. Homogeneity with respect to the target outcome variable increases the relative efficiency of the study by reducing the variance to which intervention-induced variance is compared. Insisting on homogeneity with respect to variables not highly related to treatment effects simply reduces the ability to generalize regarding the results and often reduces the sample size. For example, if the experimenter thinks that the treatment is useful for both boys and girls, over a wide range of IQs

692

and socioeconomic conditions and over a range of several years of age, it would not be a good idea to narrow the sample with respect to these characteristics in the interest of homogeneity. If (and only if) the variation across each of these variables is large enough, specific hypotheses about an interaction effect of treatment with any of the variables can be tested. Analyses can be done with subgroups of interest. More information is obtained with this method than with the process of excluding all but a homogeneous group of subjects. In the competition between desire for a large sample size and desire for a homogeneous sample, large sample size should often win. In summary, I applaud the drive toward quality methodology fostered by Barrnett et al.; I would simply add a caveat that researchers and reviewers must put the greatest emphasis on those methodological features that are really the most important and avoid getting so perfectionistic about the rest of them that we discourage attempts at such research or fail to draw legitimate inferences from it. Joseph M. Strayhorn, Jr., M.D. Allegheny General Hospital Pittsburgh, Pennsylvania

REFERENCES

Barrnett, R. J., Docherty, J. P. & Frommelt, G. M. (1991), A review of child psychotherapy research since 1963. J. Am. Acad. Child Ado/esc. Psychiatry. 30:1-14. Hulley, S. B. & Cummings, S. R. (1988), Designing Clinical Research: An Epidemi%gic Approach. Baltimore: Williams & Wilkins. Kerlinger, F. N. (1973), Foundations of Behaviora/ Research, 2nd Ed. New York: Holt, Rinehart, and Winston.

Forensic Issues To the Editor: Congratulations to Dr. Herman as well as the other authors of the special section on Forensic Child Psychiatry that appeared in the November, 1990 issue of the Journal. Having been involved in the field offorensic psychiatry for many years, I found the articles clear, cogent, and timely. . I would like to comment on two areas. First of all, in the introduction of the section, Dr. Herman spoke about incidents in which forensic psychiatric intervention might be inappropriate or psychologically unsound. However, the examples that he gave were of evaluations that were either performed inadequately or which offered conclusions that went beyond the data. The situations did, however, appear to warrant forensic psychiatric evaluation. Thus, it was not the need for a forensic psychiatric evaluation that was in question but rather the expertise of the product. There are, in fact, times when one should decline to perform a forensic psychiatric evaluation. An example of this is when an attorney wants a child custody evaluation performed where the psychiatrist sees only one parent and the child. Usually, however, when an attorney requests an evaluation, it is clear that there is a need. It becomes the task of the forensic expert to decide whether he or she wishes to undertake the evaluation and, if so, whether the appropriate questions are being posed by the referral source for the given situation. Should the psychiatrist decide to participate, a thorough evaluation must be done and conclusions drawn, although the expert must be careful not

J.Am.Acad. Child Ado/esc. Psychiatry, 30:4, Ju/y 1991