Journal o f Abnormal Child Psychology, Vol. 3, No. 3, 1977
Measuring Dimensions of Deviant Behavior: The Behavior Problem Checklist Herbert C. Quay Universi~ o f Miami
Beginning with the pioneering work of Hewitt and Jenkins in 1946, a considerable body of research has clearly demonstrated that such generic labels as "delinquent," "emotionally disturbed," "socially maladjusted," and "behaviorally disordered" serve only to mask the fact that behavior supposedly subsumed by these labels is by no means homogeneous. Yet, theory, research, and remedial efforts with respect to children carrying these labels have continued to suffer from the heterogeneity of children so labeled. Clearly, more definitive specifications of the nature of the behavioral deviance under study is warranted. While investigators using multivariate statistical methods in the study of behavioral dimensions manifested by "deviant" children have not always agreed as to the exact number and nature of dimensions involved, such differences are usually accountable for in terms of preference for orthogonal versus oblique factors and "broad" versus "narrow" factors. The vast majority of factor analytic studies upon which the Behavior Problem Checklist (BPC) (see Quay & Peterson, 1975) is based were accomplished by techniques designed to produce broad and factorially independent dimensions. The Behavior Problem Checklist (BPC) is a 3-point scale for the rating of problem behavior traits occurring in childhood and adolescence. Its three primary subscales (Conduct-Problem, Personality-Problem, and Inadequacy.Immaturity) have been derived from the factor analysis of behavior ratings on both deviant and nondeviant subjects. A fourth scale (Socialized-Delinquency)is composed of items derived from factor-analytic studies of case history records; however, since these behaviors may also be directly observed in some settings they have been included in the BPC.
277 This j o u r n a l is copyrighted by Plenum. Each article is available for $7.50 f r o m Plenum Publishing C o r p o r a t i o n , 227 West 17th Street, New Y o r k , N . Y . 10011.
278
Quay DEVELOPMENT OF THE SUBSCALES
The Conduct-Problem, Personality-Problem and Inadequacy-Immaturity Subseales The BPC grew out of a series of investigations into the factor structure of behavior problems in children and adolescents. Selection of the initial item pool and the first factor analysis was accomplished by Peterson (1961), whose work served as a model for later studies. Peterson began by considering carefully the need for an adequate sampiing of the many behaviors of children that can be considered as deviant. This sampling was an important step since behavior traits not included obviously could not appear in any syndromes that might subsequently be isolated. Over 400 representatively selected case folders from the files of a child guidance clinic were inspected and the referral problern~ of each child were noted. Eliminating overlap and selecting items on the basis of relatively frequent occurrence, 58 items descriptive of deviant behavior were chosen and compiled into a checklist. Students in kindergarten through sixth grade (iV = 831) were then rated by their teachers on these items. The intercorrelations among all the items were obtained and this matrix was subjected to orthogonal factor analysis. Two factors emerged which Peterson called Conduct Problem (CP) and Personality Problem (PP). A subsequent study of seventh- and eighth-graders isolated another orthogonal dimension labeled Inadequacy-Immaturity (Ii) (Quay & Quay, 1965). Additional factor-analytic studies have provided replications of the three major dimensions in samples of juvenile delinquents (Quay, 1964a, 1966), students in public school classes for the emotionally disturbed (Quay, Morse, & Cutler, 1966), children seen in child guidance clinics (Peterson, Becker, Shoemaker, Luria, & Hellmer, 1961; Lessing & Zagorin, 1971), and children with learning disabilities (Paraskevopoulos & McCarthy, 1970).
The Socialized Delinquency Subseale The six items comprising the Socialized Delinquency subscale (SD) appeared as a factor in studies of the case history records of juvenile delinquents (Hewitt & Jenkins, 1946; Quay, 1964b, 1966). Subsequently, Brady (1970) demonstrated the homogeneity of this subscale in his factor analysis of BPC ratings of emotionally disturbed children in public schools. Nevertheless, the data with respect to the reliability and validity of this scale are much more limited and its use should be tempered by that fact.
Behavior Problem Checklist
279
INTERCORRELATIONS AMONG THE SUBSCALES The underlying factor dimensions measured by the subscale items are orthogonal. However, since the subscales actually provide only estimates of the true factor scores on the four dimensions they can be, and frequently are, intercorrelated, lntercorrelations range from .09 (SD vs. PP) to .72 (II vs. PP) and are highly variable between samples on which they have been .computed. They seem not to be particularly related to the nature (deviant vs. normal) of the sample. It is clear that the CP, PP, and SD scales are quite independent. Inadequacy-Immaturity relates to CP and PP in varying degrees, with the observed interrelationship accounting for as little as 4% of the variance and as great as 52%.
Content Validity To demonstrate content validity, it must be shown that test items are a representative sample of behaviors in the domain under study. The method whereby items were originally selected for the BPC provides evidence for the content validity of the BPC with respect to the domain of deviant behaviors of children and adolescents.
Concurrent Validity There are a number of criteria which provide relevant tests with respect to the concurrent validity of the BPC. These include the relationship of the BPC scores to behavior disorder, of both general and more specific nature, as defined by other means - usually the judgments of clinicians or teachers. Speer (1971) obtained BPC ratings from parents on children who were to be seen at a child guidance clinic and on siblings of such children whom the parents were inclined to rate. One hundred and seventy-three ratings were obtained on patients while 357 were obtained on siblings. In addition, 445 children who attended five socioeconomicaUy heterogeneous schools were rated voluntarily by their parents; according to Speer this sample is best thought of as simply a nonclinic contrast group rather than as a random normal sample. In total, ratings were obtained for 975 children from 380 families. For statistical analysis the children were grouped as patients, siblings, and nonclinic by sex and by age (5-8, 9-12, 13 and older). An analysis of variance of subsample of ratings by 135 mothers and 45 fathers indicated an absence of reliable parental differences on all four scales. On the CP scale, clinic children of both sexes received more extreme scores than either the siblings or nonclinic children while the latter two groups did not differ from each other. While there were no reliable age or interaction effects for either sex, boys received significantly higher ratings overall than did girls.
280
Quay
On the PP scale, clinic children of both sexes received higher scores than did either the siblings or the nonclinic children; again the latter two groups did not differ. There were no age or interaction effects and although girls received slightly higher scores than boys, the differences were not significant. The clinic cases of both sexes were again rated more deviant than the other two groups on the II scale. As in the case of the CP and PP scales, the siblings and nonclinic children did not differ reliably. In contrast to the previous scales, all boys in the 9-to-12 and 13-and-up age ranges received higher scores than did boys in the 5-to-8 age range. Overall, boys received significantly higher scores than did girls. On the SD scale, clinic girls received higher scores than did the other two groups. For the boys, the clinic group received higher scores than did the siblings but the clinic cases did not significantly differ from the nonclinic group, although the mean differences were in the expected direction. For both boys and girls in all groups, the older age group (13 and up) received higher scores than did the 5to 12-year-old children. An overall comparison of boys and girls revealed that significantly higher scores were obtained by the boys. In summary, Speer (1971, p. 226) reported: "Obviously, the most significant finding of this study is that the BPC ratings reliably and significantly differentiate child patients and two samples of child nonpatients on the three best established and replicated factor scales; CP, PP and II. To this extent an increment of 'known groups' validity has been demonstrated." In subsequent research, Zold and Speer (1971) investigated the utility of the BPC in the measurement of the effects of therapeutic intervention in terms of whether parental ratings of child guidance clinic patients reflected changes after treatment. Four different sets of BPC ratings were obtained: (1)92 ratings from parents whose children had been seen 6 to 12 months previously, (2) 102 posttherapy ratings from parents who had also rated their children prior to therapy as part of an earlier study, (3) 102 pretherapy ratings from the parents of group 2, and (4) ratings of 445 nonclinic children, which had been obtained as part of the earlier study by Speer (1971)discussed above. Results indicated that boys in both of the two posttherapy samples were rated significantly less symptomatic than at the start of therapy on the CP, PP, and II scales. Only groups 3 and 4 differed significantly on the SD scale. The findings for girls were generally similar to those for boys, but were somewhat less reliable. According to Zold and Speer (1971, p. 523), "The most important finding in this study is that the BPC not only differentiated between nonclinic children and a group of child guidance clinic patients at i n t a k e . . , but it also reflected changes after therapy. These findings are more reliable for boys than for girls and apply mainly to the first three scales of'the BPC: CP, PP and II." The differential validity of the subscales of the BPC against the criterion of psychiatrically diagnosed syndromes of aggressive, hyperactive, and withdrawn
Behavior Problem Checklist
281
behaviors has been studied by Proger, Mann, Green, Bayuk and Burger (1975). Ratings were provided by public school special class teachers on a stratified sample of boys. The average age of the total sample was just over 10 years with a range of 8 to 14 years. Discriminant analysis indicated that a combination of all four scales could correctly identify group membership in 62 of the 95 children (65%). Given that the three major syndromes had to be obtained by collapsing more detailed psychiatric diagnoses and given the known questionable reliability of such diagnoses, the number of correct identifications is very encouraging. Of course, many of the behaviors subsumed by the BPC may have, and probably should have, influenced the clinical diagnosis. Sultana (1974) obtained ratings on three groups of children. Thirty-four children, both boys and girls, between ages 5 and 15 who were attending a special facility for emotionally disturbed children constituted the experimental group. A second group was composed of 34 children, judged normal by their classroom teachers, who came from the same pubfic school classrooms as did the disturbed children and who were matched with a child in the disturbed group for sex, race, and age. A third sample of 1,129 was composed of all children in those classrooms which had a child attending the special school for part of the school day; some of these children might have been clinically classifiable as disturbed. Each of these children was rated by his teacher as either "normally adjusted" or "poorly adjusted" or "unsure." For each of the four factor scales differences between the disturbed and matched normal groups were significant at the .01 level. By the application of cutoff scores for the four scales the large unselected sample was classified as severely disturbed (1.8%), moderately disturbed (10.4%), and mildly disturbed (17.9%). These classifications were then compared to the global judgmental classification made by the teachers. Of the 214 children so labeled by their teachers, 165 were identified as either severely, moderately, or mildly disturbed by the BPC, a rate of correct identification of 77%. At the same time, the BPC identified 107 of the 1,129 children as disturbed to some degree who were not so considered by their teachers; thus 10% of the total sample was misclassified. The application of these cutoff scores to the experimental group resulted in 77% being correctly classified as disturbed. Additional data on samples of children with other types of disorders has been provided by McCarthy and Paraskevopoulos (1969), Lowry and Campbell (1973), Campbell (1974) and Campbell and Finketstein (! 974).
Construct Validity Many of the studies which bear upon the validity of the scales of the BPC as measures of the four dimensions of deviant behavior have been undertaken
282
Quay
with a view tO studying the nature of the dimensions or psychological constructs themselves; investigating the validity of the measurement technique (the BPC) has often been peripheral to the interests of the researcher. Whatever the original purpose, all such studies do bear on the construct validity of the subscales of the BPC. The following studies have provided evidence with respect to the association between BPC subscales and other variables to which the subscales may have logically been expected to be related. Activity Level. Victor, Halverson, lnoff, and Buczkowski (1973)obtained correlations of BPC scores with both ratings of hyperactivity and mechanical recordings of activity in a series of free-play sessions. Although the sample was small (N = 8), the correlation between recorded activity and the CP scales was significant (.65). Academic Achievement. Glavin and Annesley (1971) studied academic achievement in a group of behavior problem children. While 81.5% of the total group were underachieving in reading and 72,.3% were underachieving in arithmetic, there were no significant differences between two groups classified as either CP or PP with respect to skills in both subjects. This, of course, suggests that both dimensions of behavior disorder will result in underachievement in the academic setting. Glavin and DeGirolamo (1970) studied the types of spelling errors made by children representative of the CP and PP dimensions. They found that the PP children wrote significantly more unrecognizable words while the CP children more often either refused to write or did not complete the word. These results would follow generally from the general form of social behavior exhibited by children representative of the two dimensions. Delinquent Recidivism. Mack (1969) found 65 male repeat offenders who had been rated by their parole officers to exceed a group of 59 nonrepeaters on the CP subscale. The correlation of CP with recidivism was .55, although the author reports considerable overlap in the distribution of the two groups on CP. Autonomic Nervous System Reactivity. Borkovec (1970)used the BPC to select three contrasting groups of juvenile delinquents: those high on CP, those high on PP, and those without elevated scores on either subscale. Subjects were presented with successive tone stimuli while skin conductance and heart rate were monitored. The group high on the CP subscale gave significantly lower galvanic skin responses to the initial stimulus and lower, though not significantly lower, heart rate changes and pre-post stimulus basal skin conductance increases. Results were discussed in the context of a theory relating extreme conduct problem behavior to autonomic underarousal. Preference for Novelty and Complexity. In a study designed to test the hypothesis that extreme scorers on the CP subseale are stimulation-seekers, Skrzypek (1969) contrasted high CP- and high PP-scoring delinquents on a variety of tasks and measures. He found that the high CP group had significantly lower initial anxiety and significantly higher initial novelty and complexity
Behavior Problem Checklist
283
preference scores. After a period of perceptual isolation, the high CP group increased their preferences for complexity. After a period of arousal, high PP scorers increased in anxiety and decreased in preference for complexity. Size Discrimination. Webb (1973) in an investigation of the relation between personality disorder and size discrimination, found that, consistent with his theoretical expectations, more children with high scores on the PP dimension showed deviant within-trial response variation than did normals or CP subjects. Anemia. Webb and Oski (1974) studied a group of 74 black males aged 13 and 14; of these, 38 were anemic by medical criteria while 36 were controls whose hemoglobins were within normal limits. BPC ratings were obtained from teachers unaware of the hematological condition of the children. Analysis indicated that while there were no differences in the groups of the PP and II scales, the anemic group tended to have higher CP scores. The author suggests that "a selective effect of iron deficiency may involve heightened activation manifest a restless, irritable disruptive behavior" (p. 155). Relationships with Other Measures of Behavioral Deviance. Schultz, Manton, and Salvia (1972) investigated the relationship between BPC subscales and the Bower.Lambert procedures for identifying emotionally handicapped children (Bower, 1969) in a sample of 1,656 rural third and fourth grade children. With one exception, the correlation between the BPC scales (SD was not scored) and the Bower-Lambert procedures were all significant at the .01 level but were uniformly of low magnitude; the highest relationship was only .42. However, as the authors point out, the Bower-Lambert procedures require that a certain number of children be designated as positives for emotional handicap. Some children could well have been false positives and may not have had behavior problems to any appreciable degree, thus reducing the relationship between behavior disorder as measured by the BPC and emotional handicap as defined by the Bower-Lambert measures. In the study already discussed with respect to concurrent validity, Proger et al. (1975) also related the BPC subscales to the subscales of the Devereaux Elementary School Behavior Rating Scale (Spivack & Swift, 1967). The CP subscale was significantly correlated with 7 of the 14 Devereaux scales: Classroom Disturbance (.81), Impatience (.49), Disrespect-Defiance (.81), External Blame (.45), Irrelevant Responsiveness (.53), Unable to Change (.23), and Quits (.27). The PP subscale was significantly correlated with Achievement Anxiety (.30), Inattentive-Withdrawn (.37), UnalSle to Change (.27), Quits (.23), and Slow Work (.21). The II subscale related to External Reliance (.31), Comprehension (-.27), Inattentive-Withdrawn (.50), Unable to Change (.28), Quits (.34), and Slow Work (.46). The SD scale correlated significantly with only two of the Devereaux Subscales: Classroom Disturbance (.23) and Disrespect-Defiance (.21 ). With the exception of the significant, but very low, correlations of CP, PP, and II with the Unable to Change, Quits, and Slow Work Devereaux subscales,
284
Quay
the differential pattern of correlations is highly supportive of the construct validity of the four BPC subscales. " The studies described above all provide evidence for the relationships of the BPC subscale to a wide variety of psychologically relevant variables which are consonant with the psychological meaning of the subscales.
RELIABILITY
Internal Consistency The factorially homogeneous nature of the subscales could be expected to result in high internal consistency reliability. In a sample of over 1,000 delinquents who had been rated by correctional staff, Quay and Parsons (1972) found KR20 reliabilities of .89 for CP, .83 for PP, ana .68 for II. Mack (1969) reported split-half reliabilities of .92 for CP, .81 for PP, and .26 for II in a sample of 124 juvenile delinquents.
lnterrater Reliability A variety of studies have provided data on the correlations between different raters on various samples of children and adolescents. In his original research, Peterson (1961 ) found that in a sample of 126 kindergarten children the correlations between teachers were .77 for CP and .75 for PP. Quay and Quay (1965) found much lower teacher agreement for seventh and eighth grade groups: .71 and .58 for CP, .31 and .22 for PP. However, the teachers completing these ratings averaged only 1 hour per day of contact with the adolescents whom they rated. Noffsinger (1969) obtained interrater reliabilities of .83 for CP and .61 for PP in a sample of elementary age emotionally disturbed children. Quay, Sprague, Shulman, and Miller (1966) obtained ratings from mothers, fathers, and teachers on children who were clients of a child guidance clinic. The mterparent correlations were .78 for CP and .67 for PP. Mother-teacher correlations were much lower: .33 for CP and .41 for PP, while the father-teacher correlations were .23 for CP and .32 for PP. While parents seemed to agree fairly well, there was obviously little agreement between parents and teachers. As might be expected, the degree of agreement between raters is a function of who the raters are that are being compared and the situations in which the respective raters make their observations. Obviously, parent.parent agreement and teacher-teacher agreement is higher than parent-teacher agreement. It is also not surprising that the easier-to.observe behaviors of the CP scale generally result in higher reliabilitieso
Behavior Problem Checklist
285
Test-R etest R el&bility
The degree to which ratings made at different times could be expected to be in agreement is a function of the length of intervening intervals and the nature of events during the interval. Ideally, there should be an interval long enough to minimize the possible influence of memory yet short enough to minimize the effects of intervening events on the behavior of those being rated. Evans (1975) obtained repeated ratings over a 2-week interval on innercity fourth grade children (46 males and 51 females) chosen at random from 51 classrooms. The correlations between the first and second administrations for the boys were .85 (CP), .74 (PP), .82 (II), and .82 (SD). For the girls, the correlations were .91 (CP), .87 (PP), .93 (~,) and .79 (SD). Victor and Halverson (1976) obtained stability coefficients over a 2-year interval on a sample of 23 males and 25 females in grades 1,2, and 3 at the time of the original ratings. Coefficients for the males were .57 for CP, .28 for PP, and .0_ for II. For females, the correlations were .74 for CP, .35 for PP, and .50 for II. With respect to the longer intervals, it should be noted that "emotional disturbance" in school children has been shown to be relatively transitory (Glavin, 1972). USES OF THE BEHAVIOR PROBLEM CHECKLIST
The identification of, and differentiation among, dimensions of deviance in children and adolescence is the most useful purpose to which the BPC, and has been, put. In treatment settings this has involved selecting children for differing interventions based, at least in part, on their subscale scores (Gerard, 1970; lngram, Gerard, Quay, & Levinson, 1970; Tupker & Pointer, 1975). Investigators have also utilized the BPC in the measurement of behavior change in response to drugs (e.g., Millichap, Aymat, Sturgis, Larsen, & Egan, 1968; Knights & Hinton, 1969; Greenwold & Jones, 1971) and to psychological interventions (e.g., Zold & Speer, 1971 ; Aksamit, 1974; Brown, 1975). In the research setting, the primary question has usually been that of how children who are representative of one dimension differ from those representative of another in systematic and psychologically relevant ways. The subscales have been found highly useful in selecting subjects for research of this type, as we have noted in the discussion of construct validity. A major advantage of describing research samples of deviant children in terms of the dimensions of the BPC is that such description facilitates comparisons of the results among research studies utilizing different methods of treatment or experimentation, whether psychological, educational, medical, correctional, or laboratory-experimental. Comparing the results of studies utilizing samples described for example as simply "emotionally disturbed," "delinquent," "hyperactive" is next to impossible because of the tremendous behavioral variability subsumed by such descriptions.
286
Quay
REFERENCES Aksamit, D. L. Identification and change in the behavior o f students placed in ~pecial classes for the emotionally disturbed. Unpublished doctoral dissertation, University of Nebraska, 1974. Borkovec, T. D. Autonomic reactivity to sensory stimulation in psychopathic, neurotic and normal juvenile delinquents. Journal o f Consulting and Clinical Psychology, 1970, 35, 217-222. Bower, E. M. Early identification of emotionally handicapped children in school (2nd ed.). Springfield, Illinois: Charles C Thomas, 1969. Brady, R. C. Effects of success and failure on impuisivity and distractibility o f three types o f educationally handicapped children. Unpublished doctoral dissertation, University of Southern California, 1970. Brown, J. E. A comparison o f social casework and behavioral contracting with juvenile delinquents on probation. Unpublished master's thesis, University of Calgary, 1975. Campbell, S. B. Cognitive styles and behavior problems of clinic boys: A comparison of epileptic, hyperactive, learning disabled, and normal g~oups. Journal of Abnormal ChildPsychoiogy, 1974, 2, 307-312. Campbell, S. B., & Finkelstein, J. A comparison of mother-child interaction patterns in learning disabled and normal boys. Paper presented at the Eighteenth International Congress of Applied Psychology, Montreal, July 21-August 2, 1974. Evans, W. R. The Behavior Problem Checklist: Data from an inner-city population.Psychology in the Schools, 1975, 12, 301-303. Gerard, R. Institutional innovations in juvenile corrections. FederalProbation, 1970, 34(Jan.), 37-44. Glavin, J. P. Persistence of behavior disorder in children. Exceptional Children, 1972, 38, 367-376. Glavin, J. P., & Annesley, F. R. Reading and arithmetic correlates of conduct problem and withdrawn children. Journal of Special Education, 1971,5, 213-219. Glavin, J. P., & DeGitolamo, G. Spelling errors of withdrawn and conduct problem children. Journal of Special Education, 1970, 4, 119-204. Greenwold, W. E., & Jones, P. R. The effect of methylphenidate on behavior of three school children: A pilot investigation. Exceptional Children, 1971,38, 261-262. Hewitt, L. E., & Jenkins, R. L. Fundamental patterns o f maladjustment: The dynamics o f their origin. Springfield: State of IRinois, 1946. Ingrain, G. L., Gerard, R. E., Quay, H. C., & Levinson, R. B. An experimental program for the psychopathic delinquent. Journal o f Research in Crime and Delinquency, 1970, Jan., 24-30. Knights, R, M., & Hinton, G. G. The effects of methylphenidate (ritalin) on the motor skills and behavior of children with learning problems. Journal of Nervous and Mental Disease, 1969, 148, 643-653. Lessing, E. E., & Zagorin, S. W. Dimensions of psychopathology in middle childhood as evaluated by three symptom checklists. Educational and PsychologicalMeasurement, 1971,31, 175-197. Lowry, C. B., & Campbell, S. B. Some psychological correlates of generalized and focal epilepsy in children. Journal o f Abnormal Child Psychology, 1973, 1, 267-279. Mack, J. L. Behavior ratings on recidivist and non-recidivist delinquent males. Psychological Reports, 1969, 25, 260. McCarthy, J. M., & Paraskevopoulos, J. Behavior patterns of learning disabled, emotionally disturbed, and average children. Exceptional Children, 1969, 36, 69-74. Milllchap, J. G., Aymat, F., Stuxgis, L., Larsen, K. W., & Egan, R. Hyperkinetic behavior and learning disorders. American Journal of Diseases of Children, 1968, 116, 235-244. Noffsinger, T. L. Proposal for a doctoral thesis, George Peabody College, 1969. Paxaskevopoulos, J., & McCarthy, J. M. Behavior patterns of children with special learning disabilities.Psychology in the Schools, 1970, 7, 42-46. Peterson, D. R. Behavior problems of middle childhood. Journal of Consulting Psychology, 1961,25, 205-209.
Behavior Problem Checklist
287
Peterson, D. R., Becket, W. C., Shoemaker, D. J,, Luria, Z., & Hellmer, L. A. Child behavior problems and parental attitudes. Child Development, 1961,32, 151-162. Proger, B. B., Mann, L., Green, P. A., Bayuk, R. J., Jr., & Burger, R. M. Discriminators of clinically defined emotional maladjustment: Predictive validities of the Behavior Problem Checklist and Devereaux Scales. Journal of Abnormal Child Psychology., 1975,3, 71-82. Quay, H. C. Personality dimensions in delinquent males as inferred from the factor analysis of behavior ratings. Journal of Research in Crime and Delinquency, 1964,1, 33-37. (a) Quay, H. C. Dimensions of personality in delinquent boys as inferred from the factor analysis of case history data. Child Development, 1964, 35, 477-484. (b) Quay, H. C. Personality patterns in preadolescent delinquent boys. Educational and Psy. chological Measurement. 1966, 16, 99-110. Quay, H. C., & Parsons, C. B. The differential behavioral classification of the juvenile offender (2nd ed.). Washington: U.S. Bureau of Prisons, 1972. Quay, H. C., & Peterson, D. R. Manual for the Behavior Problem Checklist. Unpublished manuscript, 1975. Quay, H. C., & Quay, L. C. Behavior problems in early adolescence. Child Development, 1965,36, 215-220. Quay, H. C., Sprague, R. C., Shulman, H. S., & Miller, A. L. Some correlates of personality disorder and conduct disorder in a child guidance clinic sample. Psychology in the Schools, 1966,3, 44-47. Quay, H. C., Morse, W. C., & Cutler, R. L. Personality patterns of pupils in special classes for the emotional disturbed. Exceptional Children, 1966, 32, 297-301. Schultz, E. W., Manton, A. B., & Salvia, J. A. Screening emotionally disturbed children in a rural setting. Exceptional Children, 1972, 38, 134-137. Skrzypek, G. J. Effect of perceptual isolation and arousal on anxiety, complexity preference, and novelty preference in psychopathic and neurotic delinquents.Journal of Abnormal Psychology, 1969, 74. 321-329. Speer, D. C. The Behavior Problem Checklist (Peterson-Quay): Baseline data from parents of child guidance and nonclinic children. Journal of Consulting and ClinicalPsychology, 1971,36, 221-228. Spivack, G. and Swift, J. Devereaux Elementary School Behavior Rating Scale Manual. Devon, Pennsylvania: Devereaux Foundation Press, t 967. Sultana, Q. An analysis of the Quay-Peterson Behavior Problem Checklist as an instrument to screen emotionally disturbed children. Unpublished doctoral dissertation, Universit3' of Georgia, 1974. Victor, J. B., & Halverson. C. F., Jr. Behavior problems in elementary school children: A follow-up study. Journal of Abnormal Child Psychology, 1976, 4, 17-29. Victor, J. B., Halverson, C. F., Jr., lnoff, G., & Buczkowski, H. J. Objective behavior measures of first and second grade boys' free play and teachers' ratings on a behavior problem checklist. Psychology in the Schools, 1973, l O, 439-443. Webb, T. E. Stereoscopic size discrimination in disturbed and nondisturbed children. Perceptual and Motor Skills, 1973, 36, 323-328. Webb, T. E., & Oski, F. A. Behavioral status of young adolescents with iron deficiency anemia. Journal of Special Education, 1974, 8, 153-156. Zold, A. C., & Speer, D. C. Follow-up study of child guidance patients by means of the Behavior Problem Checklist. Journal of Clinical Psychology, 1971, 27, 519-524.
Measuring Dimensions of Deviant Behavior: The Behavior Problem Checklist Herbert C. Quay Universi~ o f Miami
Beginning with the pioneering work of Hewitt and Jenkins in 1946, a considerable body of research has clearly demonstrated that such generic labels as "delinquent," "emotionally disturbed," "socially maladjusted," and "behaviorally disordered" serve only to mask the fact that behavior supposedly subsumed by these labels is by no means homogeneous. Yet, theory, research, and remedial efforts with respect to children carrying these labels have continued to suffer from the heterogeneity of children so labeled. Clearly, more definitive specifications of the nature of the behavioral deviance under study is warranted. While investigators using multivariate statistical methods in the study of behavioral dimensions manifested by "deviant" children have not always agreed as to the exact number and nature of dimensions involved, such differences are usually accountable for in terms of preference for orthogonal versus oblique factors and "broad" versus "narrow" factors. The vast majority of factor analytic studies upon which the Behavior Problem Checklist (BPC) (see Quay & Peterson, 1975) is based were accomplished by techniques designed to produce broad and factorially independent dimensions. The Behavior Problem Checklist (BPC) is a 3-point scale for the rating of problem behavior traits occurring in childhood and adolescence. Its three primary subscales (Conduct-Problem, Personality-Problem, and Inadequacy.Immaturity) have been derived from the factor analysis of behavior ratings on both deviant and nondeviant subjects. A fourth scale (Socialized-Delinquency)is composed of items derived from factor-analytic studies of case history records; however, since these behaviors may also be directly observed in some settings they have been included in the BPC.
277 This j o u r n a l is copyrighted by Plenum. Each article is available for $7.50 f r o m Plenum Publishing C o r p o r a t i o n , 227 West 17th Street, New Y o r k , N . Y . 10011.
278
Quay DEVELOPMENT OF THE SUBSCALES
The Conduct-Problem, Personality-Problem and Inadequacy-Immaturity Subseales The BPC grew out of a series of investigations into the factor structure of behavior problems in children and adolescents. Selection of the initial item pool and the first factor analysis was accomplished by Peterson (1961), whose work served as a model for later studies. Peterson began by considering carefully the need for an adequate sampiing of the many behaviors of children that can be considered as deviant. This sampling was an important step since behavior traits not included obviously could not appear in any syndromes that might subsequently be isolated. Over 400 representatively selected case folders from the files of a child guidance clinic were inspected and the referral problern~ of each child were noted. Eliminating overlap and selecting items on the basis of relatively frequent occurrence, 58 items descriptive of deviant behavior were chosen and compiled into a checklist. Students in kindergarten through sixth grade (iV = 831) were then rated by their teachers on these items. The intercorrelations among all the items were obtained and this matrix was subjected to orthogonal factor analysis. Two factors emerged which Peterson called Conduct Problem (CP) and Personality Problem (PP). A subsequent study of seventh- and eighth-graders isolated another orthogonal dimension labeled Inadequacy-Immaturity (Ii) (Quay & Quay, 1965). Additional factor-analytic studies have provided replications of the three major dimensions in samples of juvenile delinquents (Quay, 1964a, 1966), students in public school classes for the emotionally disturbed (Quay, Morse, & Cutler, 1966), children seen in child guidance clinics (Peterson, Becker, Shoemaker, Luria, & Hellmer, 1961; Lessing & Zagorin, 1971), and children with learning disabilities (Paraskevopoulos & McCarthy, 1970).
The Socialized Delinquency Subseale The six items comprising the Socialized Delinquency subscale (SD) appeared as a factor in studies of the case history records of juvenile delinquents (Hewitt & Jenkins, 1946; Quay, 1964b, 1966). Subsequently, Brady (1970) demonstrated the homogeneity of this subscale in his factor analysis of BPC ratings of emotionally disturbed children in public schools. Nevertheless, the data with respect to the reliability and validity of this scale are much more limited and its use should be tempered by that fact.
Behavior Problem Checklist
279
INTERCORRELATIONS AMONG THE SUBSCALES The underlying factor dimensions measured by the subscale items are orthogonal. However, since the subscales actually provide only estimates of the true factor scores on the four dimensions they can be, and frequently are, intercorrelated, lntercorrelations range from .09 (SD vs. PP) to .72 (II vs. PP) and are highly variable between samples on which they have been .computed. They seem not to be particularly related to the nature (deviant vs. normal) of the sample. It is clear that the CP, PP, and SD scales are quite independent. Inadequacy-Immaturity relates to CP and PP in varying degrees, with the observed interrelationship accounting for as little as 4% of the variance and as great as 52%.
Content Validity To demonstrate content validity, it must be shown that test items are a representative sample of behaviors in the domain under study. The method whereby items were originally selected for the BPC provides evidence for the content validity of the BPC with respect to the domain of deviant behaviors of children and adolescents.
Concurrent Validity There are a number of criteria which provide relevant tests with respect to the concurrent validity of the BPC. These include the relationship of the BPC scores to behavior disorder, of both general and more specific nature, as defined by other means - usually the judgments of clinicians or teachers. Speer (1971) obtained BPC ratings from parents on children who were to be seen at a child guidance clinic and on siblings of such children whom the parents were inclined to rate. One hundred and seventy-three ratings were obtained on patients while 357 were obtained on siblings. In addition, 445 children who attended five socioeconomicaUy heterogeneous schools were rated voluntarily by their parents; according to Speer this sample is best thought of as simply a nonclinic contrast group rather than as a random normal sample. In total, ratings were obtained for 975 children from 380 families. For statistical analysis the children were grouped as patients, siblings, and nonclinic by sex and by age (5-8, 9-12, 13 and older). An analysis of variance of subsample of ratings by 135 mothers and 45 fathers indicated an absence of reliable parental differences on all four scales. On the CP scale, clinic children of both sexes received more extreme scores than either the siblings or nonclinic children while the latter two groups did not differ from each other. While there were no reliable age or interaction effects for either sex, boys received significantly higher ratings overall than did girls.
280
Quay
On the PP scale, clinic children of both sexes received higher scores than did either the siblings or the nonclinic children; again the latter two groups did not differ. There were no age or interaction effects and although girls received slightly higher scores than boys, the differences were not significant. The clinic cases of both sexes were again rated more deviant than the other two groups on the II scale. As in the case of the CP and PP scales, the siblings and nonclinic children did not differ reliably. In contrast to the previous scales, all boys in the 9-to-12 and 13-and-up age ranges received higher scores than did boys in the 5-to-8 age range. Overall, boys received significantly higher scores than did girls. On the SD scale, clinic girls received higher scores than did the other two groups. For the boys, the clinic group received higher scores than did the siblings but the clinic cases did not significantly differ from the nonclinic group, although the mean differences were in the expected direction. For both boys and girls in all groups, the older age group (13 and up) received higher scores than did the 5to 12-year-old children. An overall comparison of boys and girls revealed that significantly higher scores were obtained by the boys. In summary, Speer (1971, p. 226) reported: "Obviously, the most significant finding of this study is that the BPC ratings reliably and significantly differentiate child patients and two samples of child nonpatients on the three best established and replicated factor scales; CP, PP and II. To this extent an increment of 'known groups' validity has been demonstrated." In subsequent research, Zold and Speer (1971) investigated the utility of the BPC in the measurement of the effects of therapeutic intervention in terms of whether parental ratings of child guidance clinic patients reflected changes after treatment. Four different sets of BPC ratings were obtained: (1)92 ratings from parents whose children had been seen 6 to 12 months previously, (2) 102 posttherapy ratings from parents who had also rated their children prior to therapy as part of an earlier study, (3) 102 pretherapy ratings from the parents of group 2, and (4) ratings of 445 nonclinic children, which had been obtained as part of the earlier study by Speer (1971)discussed above. Results indicated that boys in both of the two posttherapy samples were rated significantly less symptomatic than at the start of therapy on the CP, PP, and II scales. Only groups 3 and 4 differed significantly on the SD scale. The findings for girls were generally similar to those for boys, but were somewhat less reliable. According to Zold and Speer (1971, p. 523), "The most important finding in this study is that the BPC not only differentiated between nonclinic children and a group of child guidance clinic patients at i n t a k e . . , but it also reflected changes after therapy. These findings are more reliable for boys than for girls and apply mainly to the first three scales of'the BPC: CP, PP and II." The differential validity of the subscales of the BPC against the criterion of psychiatrically diagnosed syndromes of aggressive, hyperactive, and withdrawn
Behavior Problem Checklist
281
behaviors has been studied by Proger, Mann, Green, Bayuk and Burger (1975). Ratings were provided by public school special class teachers on a stratified sample of boys. The average age of the total sample was just over 10 years with a range of 8 to 14 years. Discriminant analysis indicated that a combination of all four scales could correctly identify group membership in 62 of the 95 children (65%). Given that the three major syndromes had to be obtained by collapsing more detailed psychiatric diagnoses and given the known questionable reliability of such diagnoses, the number of correct identifications is very encouraging. Of course, many of the behaviors subsumed by the BPC may have, and probably should have, influenced the clinical diagnosis. Sultana (1974) obtained ratings on three groups of children. Thirty-four children, both boys and girls, between ages 5 and 15 who were attending a special facility for emotionally disturbed children constituted the experimental group. A second group was composed of 34 children, judged normal by their classroom teachers, who came from the same pubfic school classrooms as did the disturbed children and who were matched with a child in the disturbed group for sex, race, and age. A third sample of 1,129 was composed of all children in those classrooms which had a child attending the special school for part of the school day; some of these children might have been clinically classifiable as disturbed. Each of these children was rated by his teacher as either "normally adjusted" or "poorly adjusted" or "unsure." For each of the four factor scales differences between the disturbed and matched normal groups were significant at the .01 level. By the application of cutoff scores for the four scales the large unselected sample was classified as severely disturbed (1.8%), moderately disturbed (10.4%), and mildly disturbed (17.9%). These classifications were then compared to the global judgmental classification made by the teachers. Of the 214 children so labeled by their teachers, 165 were identified as either severely, moderately, or mildly disturbed by the BPC, a rate of correct identification of 77%. At the same time, the BPC identified 107 of the 1,129 children as disturbed to some degree who were not so considered by their teachers; thus 10% of the total sample was misclassified. The application of these cutoff scores to the experimental group resulted in 77% being correctly classified as disturbed. Additional data on samples of children with other types of disorders has been provided by McCarthy and Paraskevopoulos (1969), Lowry and Campbell (1973), Campbell (1974) and Campbell and Finketstein (! 974).
Construct Validity Many of the studies which bear upon the validity of the scales of the BPC as measures of the four dimensions of deviant behavior have been undertaken
282
Quay
with a view tO studying the nature of the dimensions or psychological constructs themselves; investigating the validity of the measurement technique (the BPC) has often been peripheral to the interests of the researcher. Whatever the original purpose, all such studies do bear on the construct validity of the subscales of the BPC. The following studies have provided evidence with respect to the association between BPC subscales and other variables to which the subscales may have logically been expected to be related. Activity Level. Victor, Halverson, lnoff, and Buczkowski (1973)obtained correlations of BPC scores with both ratings of hyperactivity and mechanical recordings of activity in a series of free-play sessions. Although the sample was small (N = 8), the correlation between recorded activity and the CP scales was significant (.65). Academic Achievement. Glavin and Annesley (1971) studied academic achievement in a group of behavior problem children. While 81.5% of the total group were underachieving in reading and 72,.3% were underachieving in arithmetic, there were no significant differences between two groups classified as either CP or PP with respect to skills in both subjects. This, of course, suggests that both dimensions of behavior disorder will result in underachievement in the academic setting. Glavin and DeGirolamo (1970) studied the types of spelling errors made by children representative of the CP and PP dimensions. They found that the PP children wrote significantly more unrecognizable words while the CP children more often either refused to write or did not complete the word. These results would follow generally from the general form of social behavior exhibited by children representative of the two dimensions. Delinquent Recidivism. Mack (1969) found 65 male repeat offenders who had been rated by their parole officers to exceed a group of 59 nonrepeaters on the CP subscale. The correlation of CP with recidivism was .55, although the author reports considerable overlap in the distribution of the two groups on CP. Autonomic Nervous System Reactivity. Borkovec (1970)used the BPC to select three contrasting groups of juvenile delinquents: those high on CP, those high on PP, and those without elevated scores on either subscale. Subjects were presented with successive tone stimuli while skin conductance and heart rate were monitored. The group high on the CP subscale gave significantly lower galvanic skin responses to the initial stimulus and lower, though not significantly lower, heart rate changes and pre-post stimulus basal skin conductance increases. Results were discussed in the context of a theory relating extreme conduct problem behavior to autonomic underarousal. Preference for Novelty and Complexity. In a study designed to test the hypothesis that extreme scorers on the CP subseale are stimulation-seekers, Skrzypek (1969) contrasted high CP- and high PP-scoring delinquents on a variety of tasks and measures. He found that the high CP group had significantly lower initial anxiety and significantly higher initial novelty and complexity
Behavior Problem Checklist
283
preference scores. After a period of perceptual isolation, the high CP group increased their preferences for complexity. After a period of arousal, high PP scorers increased in anxiety and decreased in preference for complexity. Size Discrimination. Webb (1973) in an investigation of the relation between personality disorder and size discrimination, found that, consistent with his theoretical expectations, more children with high scores on the PP dimension showed deviant within-trial response variation than did normals or CP subjects. Anemia. Webb and Oski (1974) studied a group of 74 black males aged 13 and 14; of these, 38 were anemic by medical criteria while 36 were controls whose hemoglobins were within normal limits. BPC ratings were obtained from teachers unaware of the hematological condition of the children. Analysis indicated that while there were no differences in the groups of the PP and II scales, the anemic group tended to have higher CP scores. The author suggests that "a selective effect of iron deficiency may involve heightened activation manifest a restless, irritable disruptive behavior" (p. 155). Relationships with Other Measures of Behavioral Deviance. Schultz, Manton, and Salvia (1972) investigated the relationship between BPC subscales and the Bower.Lambert procedures for identifying emotionally handicapped children (Bower, 1969) in a sample of 1,656 rural third and fourth grade children. With one exception, the correlation between the BPC scales (SD was not scored) and the Bower-Lambert procedures were all significant at the .01 level but were uniformly of low magnitude; the highest relationship was only .42. However, as the authors point out, the Bower-Lambert procedures require that a certain number of children be designated as positives for emotional handicap. Some children could well have been false positives and may not have had behavior problems to any appreciable degree, thus reducing the relationship between behavior disorder as measured by the BPC and emotional handicap as defined by the Bower-Lambert measures. In the study already discussed with respect to concurrent validity, Proger et al. (1975) also related the BPC subscales to the subscales of the Devereaux Elementary School Behavior Rating Scale (Spivack & Swift, 1967). The CP subscale was significantly correlated with 7 of the 14 Devereaux scales: Classroom Disturbance (.81), Impatience (.49), Disrespect-Defiance (.81), External Blame (.45), Irrelevant Responsiveness (.53), Unable to Change (.23), and Quits (.27). The PP subscale was significantly correlated with Achievement Anxiety (.30), Inattentive-Withdrawn (.37), UnalSle to Change (.27), Quits (.23), and Slow Work (.21). The II subscale related to External Reliance (.31), Comprehension (-.27), Inattentive-Withdrawn (.50), Unable to Change (.28), Quits (.34), and Slow Work (.46). The SD scale correlated significantly with only two of the Devereaux Subscales: Classroom Disturbance (.23) and Disrespect-Defiance (.21 ). With the exception of the significant, but very low, correlations of CP, PP, and II with the Unable to Change, Quits, and Slow Work Devereaux subscales,
284
Quay
the differential pattern of correlations is highly supportive of the construct validity of the four BPC subscales. " The studies described above all provide evidence for the relationships of the BPC subscale to a wide variety of psychologically relevant variables which are consonant with the psychological meaning of the subscales.
RELIABILITY
Internal Consistency The factorially homogeneous nature of the subscales could be expected to result in high internal consistency reliability. In a sample of over 1,000 delinquents who had been rated by correctional staff, Quay and Parsons (1972) found KR20 reliabilities of .89 for CP, .83 for PP, ana .68 for II. Mack (1969) reported split-half reliabilities of .92 for CP, .81 for PP, and .26 for II in a sample of 124 juvenile delinquents.
lnterrater Reliability A variety of studies have provided data on the correlations between different raters on various samples of children and adolescents. In his original research, Peterson (1961 ) found that in a sample of 126 kindergarten children the correlations between teachers were .77 for CP and .75 for PP. Quay and Quay (1965) found much lower teacher agreement for seventh and eighth grade groups: .71 and .58 for CP, .31 and .22 for PP. However, the teachers completing these ratings averaged only 1 hour per day of contact with the adolescents whom they rated. Noffsinger (1969) obtained interrater reliabilities of .83 for CP and .61 for PP in a sample of elementary age emotionally disturbed children. Quay, Sprague, Shulman, and Miller (1966) obtained ratings from mothers, fathers, and teachers on children who were clients of a child guidance clinic. The mterparent correlations were .78 for CP and .67 for PP. Mother-teacher correlations were much lower: .33 for CP and .41 for PP, while the father-teacher correlations were .23 for CP and .32 for PP. While parents seemed to agree fairly well, there was obviously little agreement between parents and teachers. As might be expected, the degree of agreement between raters is a function of who the raters are that are being compared and the situations in which the respective raters make their observations. Obviously, parent.parent agreement and teacher-teacher agreement is higher than parent-teacher agreement. It is also not surprising that the easier-to.observe behaviors of the CP scale generally result in higher reliabilitieso
Behavior Problem Checklist
285
Test-R etest R el&bility
The degree to which ratings made at different times could be expected to be in agreement is a function of the length of intervening intervals and the nature of events during the interval. Ideally, there should be an interval long enough to minimize the possible influence of memory yet short enough to minimize the effects of intervening events on the behavior of those being rated. Evans (1975) obtained repeated ratings over a 2-week interval on innercity fourth grade children (46 males and 51 females) chosen at random from 51 classrooms. The correlations between the first and second administrations for the boys were .85 (CP), .74 (PP), .82 (II), and .82 (SD). For the girls, the correlations were .91 (CP), .87 (PP), .93 (~,) and .79 (SD). Victor and Halverson (1976) obtained stability coefficients over a 2-year interval on a sample of 23 males and 25 females in grades 1,2, and 3 at the time of the original ratings. Coefficients for the males were .57 for CP, .28 for PP, and .0_ for II. For females, the correlations were .74 for CP, .35 for PP, and .50 for II. With respect to the longer intervals, it should be noted that "emotional disturbance" in school children has been shown to be relatively transitory (Glavin, 1972). USES OF THE BEHAVIOR PROBLEM CHECKLIST
The identification of, and differentiation among, dimensions of deviance in children and adolescence is the most useful purpose to which the BPC, and has been, put. In treatment settings this has involved selecting children for differing interventions based, at least in part, on their subscale scores (Gerard, 1970; lngram, Gerard, Quay, & Levinson, 1970; Tupker & Pointer, 1975). Investigators have also utilized the BPC in the measurement of behavior change in response to drugs (e.g., Millichap, Aymat, Sturgis, Larsen, & Egan, 1968; Knights & Hinton, 1969; Greenwold & Jones, 1971) and to psychological interventions (e.g., Zold & Speer, 1971 ; Aksamit, 1974; Brown, 1975). In the research setting, the primary question has usually been that of how children who are representative of one dimension differ from those representative of another in systematic and psychologically relevant ways. The subscales have been found highly useful in selecting subjects for research of this type, as we have noted in the discussion of construct validity. A major advantage of describing research samples of deviant children in terms of the dimensions of the BPC is that such description facilitates comparisons of the results among research studies utilizing different methods of treatment or experimentation, whether psychological, educational, medical, correctional, or laboratory-experimental. Comparing the results of studies utilizing samples described for example as simply "emotionally disturbed," "delinquent," "hyperactive" is next to impossible because of the tremendous behavioral variability subsumed by such descriptions.
286
Quay
REFERENCES Aksamit, D. L. Identification and change in the behavior o f students placed in ~pecial classes for the emotionally disturbed. Unpublished doctoral dissertation, University of Nebraska, 1974. Borkovec, T. D. Autonomic reactivity to sensory stimulation in psychopathic, neurotic and normal juvenile delinquents. Journal o f Consulting and Clinical Psychology, 1970, 35, 217-222. Bower, E. M. Early identification of emotionally handicapped children in school (2nd ed.). Springfield, Illinois: Charles C Thomas, 1969. Brady, R. C. Effects of success and failure on impuisivity and distractibility o f three types o f educationally handicapped children. Unpublished doctoral dissertation, University of Southern California, 1970. Brown, J. E. A comparison o f social casework and behavioral contracting with juvenile delinquents on probation. Unpublished master's thesis, University of Calgary, 1975. Campbell, S. B. Cognitive styles and behavior problems of clinic boys: A comparison of epileptic, hyperactive, learning disabled, and normal g~oups. Journal of Abnormal ChildPsychoiogy, 1974, 2, 307-312. Campbell, S. B., & Finkelstein, J. A comparison of mother-child interaction patterns in learning disabled and normal boys. Paper presented at the Eighteenth International Congress of Applied Psychology, Montreal, July 21-August 2, 1974. Evans, W. R. The Behavior Problem Checklist: Data from an inner-city population.Psychology in the Schools, 1975, 12, 301-303. Gerard, R. Institutional innovations in juvenile corrections. FederalProbation, 1970, 34(Jan.), 37-44. Glavin, J. P. Persistence of behavior disorder in children. Exceptional Children, 1972, 38, 367-376. Glavin, J. P., & Annesley, F. R. Reading and arithmetic correlates of conduct problem and withdrawn children. Journal of Special Education, 1971,5, 213-219. Glavin, J. P., & DeGitolamo, G. Spelling errors of withdrawn and conduct problem children. Journal of Special Education, 1970, 4, 119-204. Greenwold, W. E., & Jones, P. R. The effect of methylphenidate on behavior of three school children: A pilot investigation. Exceptional Children, 1971,38, 261-262. Hewitt, L. E., & Jenkins, R. L. Fundamental patterns o f maladjustment: The dynamics o f their origin. Springfield: State of IRinois, 1946. Ingrain, G. L., Gerard, R. E., Quay, H. C., & Levinson, R. B. An experimental program for the psychopathic delinquent. Journal o f Research in Crime and Delinquency, 1970, Jan., 24-30. Knights, R, M., & Hinton, G. G. The effects of methylphenidate (ritalin) on the motor skills and behavior of children with learning problems. Journal of Nervous and Mental Disease, 1969, 148, 643-653. Lessing, E. E., & Zagorin, S. W. Dimensions of psychopathology in middle childhood as evaluated by three symptom checklists. Educational and PsychologicalMeasurement, 1971,31, 175-197. Lowry, C. B., & Campbell, S. B. Some psychological correlates of generalized and focal epilepsy in children. Journal o f Abnormal Child Psychology, 1973, 1, 267-279. Mack, J. L. Behavior ratings on recidivist and non-recidivist delinquent males. Psychological Reports, 1969, 25, 260. McCarthy, J. M., & Paraskevopoulos, J. Behavior patterns of learning disabled, emotionally disturbed, and average children. Exceptional Children, 1969, 36, 69-74. Milllchap, J. G., Aymat, F., Stuxgis, L., Larsen, K. W., & Egan, R. Hyperkinetic behavior and learning disorders. American Journal of Diseases of Children, 1968, 116, 235-244. Noffsinger, T. L. Proposal for a doctoral thesis, George Peabody College, 1969. Paxaskevopoulos, J., & McCarthy, J. M. Behavior patterns of children with special learning disabilities.Psychology in the Schools, 1970, 7, 42-46. Peterson, D. R. Behavior problems of middle childhood. Journal of Consulting Psychology, 1961,25, 205-209.
Behavior Problem Checklist
287
Peterson, D. R., Becket, W. C., Shoemaker, D. J,, Luria, Z., & Hellmer, L. A. Child behavior problems and parental attitudes. Child Development, 1961,32, 151-162. Proger, B. B., Mann, L., Green, P. A., Bayuk, R. J., Jr., & Burger, R. M. Discriminators of clinically defined emotional maladjustment: Predictive validities of the Behavior Problem Checklist and Devereaux Scales. Journal of Abnormal Child Psychology., 1975,3, 71-82. Quay, H. C. Personality dimensions in delinquent males as inferred from the factor analysis of behavior ratings. Journal of Research in Crime and Delinquency, 1964,1, 33-37. (a) Quay, H. C. Dimensions of personality in delinquent boys as inferred from the factor analysis of case history data. Child Development, 1964, 35, 477-484. (b) Quay, H. C. Personality patterns in preadolescent delinquent boys. Educational and Psy. chological Measurement. 1966, 16, 99-110. Quay, H. C., & Parsons, C. B. The differential behavioral classification of the juvenile offender (2nd ed.). Washington: U.S. Bureau of Prisons, 1972. Quay, H. C., & Peterson, D. R. Manual for the Behavior Problem Checklist. Unpublished manuscript, 1975. Quay, H. C., & Quay, L. C. Behavior problems in early adolescence. Child Development, 1965,36, 215-220. Quay, H. C., Sprague, R. C., Shulman, H. S., & Miller, A. L. Some correlates of personality disorder and conduct disorder in a child guidance clinic sample. Psychology in the Schools, 1966,3, 44-47. Quay, H. C., Morse, W. C., & Cutler, R. L. Personality patterns of pupils in special classes for the emotional disturbed. Exceptional Children, 1966, 32, 297-301. Schultz, E. W., Manton, A. B., & Salvia, J. A. Screening emotionally disturbed children in a rural setting. Exceptional Children, 1972, 38, 134-137. Skrzypek, G. J. Effect of perceptual isolation and arousal on anxiety, complexity preference, and novelty preference in psychopathic and neurotic delinquents.Journal of Abnormal Psychology, 1969, 74. 321-329. Speer, D. C. The Behavior Problem Checklist (Peterson-Quay): Baseline data from parents of child guidance and nonclinic children. Journal of Consulting and ClinicalPsychology, 1971,36, 221-228. Spivack, G. and Swift, J. Devereaux Elementary School Behavior Rating Scale Manual. Devon, Pennsylvania: Devereaux Foundation Press, t 967. Sultana, Q. An analysis of the Quay-Peterson Behavior Problem Checklist as an instrument to screen emotionally disturbed children. Unpublished doctoral dissertation, Universit3' of Georgia, 1974. Victor, J. B., & Halverson. C. F., Jr. Behavior problems in elementary school children: A follow-up study. Journal of Abnormal Child Psychology, 1976, 4, 17-29. Victor, J. B., Halverson, C. F., Jr., lnoff, G., & Buczkowski, H. J. Objective behavior measures of first and second grade boys' free play and teachers' ratings on a behavior problem checklist. Psychology in the Schools, 1973, l O, 439-443. Webb, T. E. Stereoscopic size discrimination in disturbed and nondisturbed children. Perceptual and Motor Skills, 1973, 36, 323-328. Webb, T. E., & Oski, F. A. Behavioral status of young adolescents with iron deficiency anemia. Journal of Special Education, 1974, 8, 153-156. Zold, A. C., & Speer, D. C. Follow-up study of child guidance patients by means of the Behavior Problem Checklist. Journal of Clinical Psychology, 1971, 27, 519-524.
