Journal of Abnormal Child Psychology, Vol. 6, No. 1, 1978, pp. 107-113
CPQ Validity: The Relationship Between Children's Personality Questionnaire Scores and Teacher Ratings 1 Jerry Willis 2 and Gail Seymour University of Western Ontario
Scores on the Children's Personality Questionnaire (CPQ) were correlated with scores from rating scale items selected by a panel o f raters measuring the same behavior as a CPQ scale. Data were obtained on a normal sample o f school children in grades three to six and on two referred groups, underachievers and chilren with behavior or social problems. The teachers o f the children were asked to complete two rating scales on each child. Correlations between rating scale scores and CPQ scores suggest that, in general, the CPQ does not measure the traits it purports to measure.
Although the Children's Personality Questionnaire (CPQ) is a popular assessment device in schools and community mental health clinics, a review of the literature pertinent to its validity indicates the need for much additional research. Currently available validity data for the CPQ are based almost exclusively on factor analysis. "The CPQ is theoretically based; its scales are relevant to the hypothesized structures of personality, and validity indicates both the goodness of the hypotheses and the adequacy of the measures of each hypothesized construct" (Cattell & Porter, 1972, p. 14). Once the factorial structure of the CPQ and other tests in the Cattell series was established, researchers proceeded with a wide range of studies designed to demonstrate relationships between scores on the tests and other general variables such as underachievement and maladjustment (Lessing & Barbera, 1973 ; Lessing & Zagofin, 1969). Several reviewers, however, have questioned Cattell's heavy
Manuscript received in final form May 25, 1977. The authors would like to express their appreciation to Deborrah Smithy for help in preparing this manuscript. 2Address all correspondence to Jerry Willis, Department of Psychology, University of British Columbia, Vancouver, B.C., Canada V6T lW5. 107 0 0 9 1 - 0 6 2 7 / 7 8 / 0 3 0 0 - 0 1 0 7 $ 0 5 . 0 0 / 0 9 1978 Plenum Publishing Corporation
108
Willis and Seymour
dependency on factor analysis as the criterion for individual scale validity (Anastasi, 1965; Layton, 1965;Jackson, 1972). In her review, Anastasi (1965) concludes: "Comparative studies of the performance of this instrument in relation to other available instruments would be desirable" (p. 122). The need for several different types of validity research was also emphasized by Layton (1965) in his review of the CPQ and by Jackson (1972) in his review of the high school version of Cattell's test. Although the CPQ does have a number of flaws, there are, in fact, no alternatives to the CPQ which are part of a series that can be used from kindergarten to high school. Without additional validity studies, however, the practitioner has little data to guide the interpretation of test score profiles. We do not know, for example, if children who are rated "excitable" or "socially bold" by the test are rated similarly by other measures. The research reported here compared scores obtained on the CPQ with ratings made by teachers.
METHOD
Sub/ects Data were obtained on school children in grades three to six in a consolidated separate school system in southern Ontario. The school system serves a heterogeneous population with a range of social classes represented. Most children came from working class homes, however. Three samples were obtained from the school population. The first represented a random sample of students stratified by grade. There were 181 students in this group, 43 in the third grade, 44 in the fourth grade, 47 in the fifth grade and 47 in the sixth grade. A second group of students was made up of all children listed as underachievers on a questionnaire completed by teachers. There were 103 students in this group, 23 in grade three, 31 in grade four, 21 in grade five, and 28 in grade six. A thizd group was made up of children the teachers listed as behavior or social problems on the questionnaire. A total of 100 students were included in this group, 27 in grade three, 23 in grade four, 25 in grade five, and 25 in grade six.
Procedure The CPQ was administered to all students in the study. In addition, teachers completed behavior ratings on the children using the Devereux Elementary School Behavior Rating Scale (Spivack & Swift, 1967) and the Wellington Behavior Rating Scale (Giles, 1976).
CPQ Validity
109
The first step in evaluating the validity of the CPQ involved categorization of the 137 items on the two behavior rating scales in terms of the 14 traits that the CPQ purports to measure. Three raters independently assigned the items to a trait. The raters were an elementary school learning disabilities teacher with 14 years of experience, a child clinical psychologist with 8 years of experience, and an undergraduate psychology student. Each item of the Wellington and the Devereux was typed on individual cards. Cattell's descriptive adjectives for each trait and the trait names were placed on a board with slots below the descriptions. The raters were given the instructions, "Read the item and then place it in the slot under the description you think that item is best trying to measure." This rating procedure produced 9 items from the Devereux and 10 from the Wellington on which there was 100% agreement as to the trait measured by the item. In addition, two of the three raters agreed on the placement of 10 more items from the Devereux and 43 from the Wellington.
Data Analysis To obtain a measure of CPQ validity, the sum of scores on the rating scale items associated with each of the 14 traits on the CPQ was correlated with the score the child obtained on the CPQ traits. In addition, summed rating scale scores were correlated with CPQ scores on scales other than the one to which the items were assigned. Thus both convergent and discriminate validity were obtained.
RESULTS Although several sets of correlations were computed, only two will be presented here since the results were quite similar for all the sets. Table I presents the correlations obtained on the normal sample of students. Items with 100% agreement were used to compute the criterion score. Columns where there are no correlations indicate that no items from the Wellington or the Devereux were assigned to the same CPQ scale by all three raters. Of 140 correlations only 4 were significant at the .01 level or higher, and only 1 of these reflected a significant correlation between a CPQ scale (intelligence) and its associated scale derived from behavior rating scale items. No other scales correlated significantly with the criterion scores. When all of the items that were assigned to a particular CPQ scale by two of the three raters were used to compute the correlations, the results were no more encouraging.
-.100 -.012 -.065 -.111 -.169 a -.102 -.062 -.112 -.076 -.097 -.046 -.112 -.043 -.032
ap < .01.
N 0 03 04
H I J
C D E F G
B
A
A
.030 .170 a .002 -.081 -.117 -.062 .042 .002 .017 -.052 -.112 -.011 .088 -.020
B
.058 .067 .022 -.036 -.011 -.003 .060 .025 -.072 -.017 -.015 -.057 -.021 -.052
C .085 .038 .073 .144 .119 .157 .042 .043 .110 .126 .157 .098 .009 .076
D .020 -.056 -.049 -.108 .079 -.060 .043 -.003 -.022 -.045 -.065 .056 .215 a .063
E
F -.117 -.126 -.132 -.112 -.050 .134 -.134 -.139 -All -.099 -.086 -.106 -.097 -.064
G -.005 .047 .075 -.041 .004 -.009 .007 .079 -.056 -.084 -.078 -.160 -.024 --.122
H
I -.104 -.175 a -.114 -.062 .036 -.042 -.113 -.065 -.102 -.012 -.062 -.013 -.040 .013
J
N
O
-.049 -.047 .090 -.129 -.141 -.100 -.007 -.048 -.072 -.070 -.061 -.016 .029 .004
Q3
Table I. Correlations Between CPQ Factors and Behavior Rating Scale Factors for Normal Students
-.115 -.124 .094 -.060 -.060 -.064 -.138 -.108 -.062 --.059 -.045 -.020 -.120 -.069
Q4
Ca
7.
B
.015 .256 a -.021 .010 -.099 .166 -.145 ,106 -.008 .027 .146 .066 -.072 ,037
A
.219 a .154 .097 -.115 -.271 a -.126 .160 ,044 -.030 .091 -.210 a -.167 .209 a -.106
ap < .01.
Q4
Q~
J N 0
H I
C D E F G
B
A
.065 .102 .135 .050 .033 ,212 a .233 a .144 -.195 -,061 .032 -.195 .117 ,007
C -.051 .111 -.124 -.010 .051 .051 -.069 -.149 .055 -.096 .058 .194 .037 .234 a
D .090 .166 .240 a .075 .116 .024 -.042 .188 -.122 .030 .058 -.102 .149 -.008
E
F -.131 -.072 -.109 .083 .092 .012 -.017 .038 .008 -.032 .107 .061 -.032 -.068
G -.035 .150 -.041 .119 .068 .272 a -.269 a .076 -.201 .120 .227 a -.033 -.003 .118
H
I -.024 -.110 -.031 -.093 .220 a .089 -.105 -.046 .168 -.013 -.014 .124 -.040 .159
J
N
0
Table II. Correlations Between CPQ Factors and Behavior Rating Scale Factors for Underachievers Q3 .106 .037 .099 -.089 -.167 -.186 .053 .093 .008 .010 -,182 -.153 .113 -.097
Q4 .018 -.108 -.044 -.266 a -.073 -.267 a .220 a -.125 .090 -.063 -.200 .115 .014 -.203
112
Willis and Seymour
Table II presents the correlations obtained on the underachiever group. As before, the items included from the behavior rating scale were those that had 100% agreement. Of 140 correlations, 16 were significant at the .01 level. Of these only 2, those for Factor A and Factor B, reflected correlations between the CPQ and behavior rating scale items assigned to them. When correlations were derived using all the items that had at least two-thirds agreement, there were 6 significant correlations, none of which reflected a relationship between a CPQ scale and its associated rating scale items. Data from the other referred group, students with behavior problems, provided no additional support for CPQ validity.
DISCUSSION
The data obtained in this study represent an attempt to deal with the criticism that the CPQ has not been shown to relate to behavior external to itself. Scores on the CPQ were correlated with ratings made by teachers using items judged to measure the same behaviors as the CPQ. The results strongly suggest that the CPQ does not measure the traits it claims to. This was true of data obtained from a normal sample of children as well as two referred groups. Without data that demonstrate that CPQ scores are associated with other measures of the target behavior, the use of such an instrument in a battery of psychological tests is questionable. The absence of a relationship between ratings and CPQ scores points out clearly the dangers of relying exclusively on a factor analytic approach in developing and defining scales. While a set of items may cluster together and may appear to contain a common thread o f meaning, there is no guarantee that the label the test developer attaches to the scale actually reflects what is being measured. In the case of the CPQ, the evidence presented here suggests that the labels do not reflect what is being measured.
REFERENCE
Anastasi, A. Review of IPAT Children's Personality Questionnaire. In O. Boros (Ed.), The sixth mental measurement yearbook. Highland Park, New Jersey: Gryphon Press, 1965. Cattell, R., & Porter, R. Handbook for the Children's Personality Questionnaire. Champaign, Illinois: Institute for Personality and Ability Testing, 1972. Giles, D. Item analysis and scale validation for the Wellington Child Behavior Rating Scale. Unpublished master's thesis, University of Western Ontario, 1976. Jackson, D. Review of IPAT edition of HSPQ. In O. Buros (Ed.), The seventh mental measurement handbook. Highland Park, New Jersey: Gryphon Press, 1972. Layton, W. Review of IPAT Children's Personality Questionnaire. In O. Buros (Ed.), The seventh mental measurement yearbook. Highland Park, New Jersey: Gryphon Press, 1965.
CPQ Validity
113
Lessing, E., & Barbera, L. Relationships among scores on the California Test of Personality, the IPAT Children's Personality Questionnaire and the IPAT High School Personality Questionnaire. Educational and Psychological Measurement, 1973, 33, 985-992. Lessing, E., & Zagorin, S. Correlation between Lorge-Thorndike IQ and Factor B of the IPAT Children's Personality Questionnaire. Psychological Reports, 1969, 24, 569-570. Spivack, G., & Swift, M. Devereux Elementary School Behavior Rating Scale. Devon, Pennsylvania: The Devereux Foundation, 1967.
CPQ Validity: The Relationship Between Children's Personality Questionnaire Scores and Teacher Ratings 1 Jerry Willis 2 and Gail Seymour University of Western Ontario
Scores on the Children's Personality Questionnaire (CPQ) were correlated with scores from rating scale items selected by a panel o f raters measuring the same behavior as a CPQ scale. Data were obtained on a normal sample o f school children in grades three to six and on two referred groups, underachievers and chilren with behavior or social problems. The teachers o f the children were asked to complete two rating scales on each child. Correlations between rating scale scores and CPQ scores suggest that, in general, the CPQ does not measure the traits it purports to measure.
Although the Children's Personality Questionnaire (CPQ) is a popular assessment device in schools and community mental health clinics, a review of the literature pertinent to its validity indicates the need for much additional research. Currently available validity data for the CPQ are based almost exclusively on factor analysis. "The CPQ is theoretically based; its scales are relevant to the hypothesized structures of personality, and validity indicates both the goodness of the hypotheses and the adequacy of the measures of each hypothesized construct" (Cattell & Porter, 1972, p. 14). Once the factorial structure of the CPQ and other tests in the Cattell series was established, researchers proceeded with a wide range of studies designed to demonstrate relationships between scores on the tests and other general variables such as underachievement and maladjustment (Lessing & Barbera, 1973 ; Lessing & Zagofin, 1969). Several reviewers, however, have questioned Cattell's heavy
Manuscript received in final form May 25, 1977. The authors would like to express their appreciation to Deborrah Smithy for help in preparing this manuscript. 2Address all correspondence to Jerry Willis, Department of Psychology, University of British Columbia, Vancouver, B.C., Canada V6T lW5. 107 0 0 9 1 - 0 6 2 7 / 7 8 / 0 3 0 0 - 0 1 0 7 $ 0 5 . 0 0 / 0 9 1978 Plenum Publishing Corporation
108
Willis and Seymour
dependency on factor analysis as the criterion for individual scale validity (Anastasi, 1965; Layton, 1965;Jackson, 1972). In her review, Anastasi (1965) concludes: "Comparative studies of the performance of this instrument in relation to other available instruments would be desirable" (p. 122). The need for several different types of validity research was also emphasized by Layton (1965) in his review of the CPQ and by Jackson (1972) in his review of the high school version of Cattell's test. Although the CPQ does have a number of flaws, there are, in fact, no alternatives to the CPQ which are part of a series that can be used from kindergarten to high school. Without additional validity studies, however, the practitioner has little data to guide the interpretation of test score profiles. We do not know, for example, if children who are rated "excitable" or "socially bold" by the test are rated similarly by other measures. The research reported here compared scores obtained on the CPQ with ratings made by teachers.
METHOD
Sub/ects Data were obtained on school children in grades three to six in a consolidated separate school system in southern Ontario. The school system serves a heterogeneous population with a range of social classes represented. Most children came from working class homes, however. Three samples were obtained from the school population. The first represented a random sample of students stratified by grade. There were 181 students in this group, 43 in the third grade, 44 in the fourth grade, 47 in the fifth grade and 47 in the sixth grade. A second group of students was made up of all children listed as underachievers on a questionnaire completed by teachers. There were 103 students in this group, 23 in grade three, 31 in grade four, 21 in grade five, and 28 in grade six. A thizd group was made up of children the teachers listed as behavior or social problems on the questionnaire. A total of 100 students were included in this group, 27 in grade three, 23 in grade four, 25 in grade five, and 25 in grade six.
Procedure The CPQ was administered to all students in the study. In addition, teachers completed behavior ratings on the children using the Devereux Elementary School Behavior Rating Scale (Spivack & Swift, 1967) and the Wellington Behavior Rating Scale (Giles, 1976).
CPQ Validity
109
The first step in evaluating the validity of the CPQ involved categorization of the 137 items on the two behavior rating scales in terms of the 14 traits that the CPQ purports to measure. Three raters independently assigned the items to a trait. The raters were an elementary school learning disabilities teacher with 14 years of experience, a child clinical psychologist with 8 years of experience, and an undergraduate psychology student. Each item of the Wellington and the Devereux was typed on individual cards. Cattell's descriptive adjectives for each trait and the trait names were placed on a board with slots below the descriptions. The raters were given the instructions, "Read the item and then place it in the slot under the description you think that item is best trying to measure." This rating procedure produced 9 items from the Devereux and 10 from the Wellington on which there was 100% agreement as to the trait measured by the item. In addition, two of the three raters agreed on the placement of 10 more items from the Devereux and 43 from the Wellington.
Data Analysis To obtain a measure of CPQ validity, the sum of scores on the rating scale items associated with each of the 14 traits on the CPQ was correlated with the score the child obtained on the CPQ traits. In addition, summed rating scale scores were correlated with CPQ scores on scales other than the one to which the items were assigned. Thus both convergent and discriminate validity were obtained.
RESULTS Although several sets of correlations were computed, only two will be presented here since the results were quite similar for all the sets. Table I presents the correlations obtained on the normal sample of students. Items with 100% agreement were used to compute the criterion score. Columns where there are no correlations indicate that no items from the Wellington or the Devereux were assigned to the same CPQ scale by all three raters. Of 140 correlations only 4 were significant at the .01 level or higher, and only 1 of these reflected a significant correlation between a CPQ scale (intelligence) and its associated scale derived from behavior rating scale items. No other scales correlated significantly with the criterion scores. When all of the items that were assigned to a particular CPQ scale by two of the three raters were used to compute the correlations, the results were no more encouraging.
-.100 -.012 -.065 -.111 -.169 a -.102 -.062 -.112 -.076 -.097 -.046 -.112 -.043 -.032
ap < .01.
N 0 03 04
H I J
C D E F G
B
A
A
.030 .170 a .002 -.081 -.117 -.062 .042 .002 .017 -.052 -.112 -.011 .088 -.020
B
.058 .067 .022 -.036 -.011 -.003 .060 .025 -.072 -.017 -.015 -.057 -.021 -.052
C .085 .038 .073 .144 .119 .157 .042 .043 .110 .126 .157 .098 .009 .076
D .020 -.056 -.049 -.108 .079 -.060 .043 -.003 -.022 -.045 -.065 .056 .215 a .063
E
F -.117 -.126 -.132 -.112 -.050 .134 -.134 -.139 -All -.099 -.086 -.106 -.097 -.064
G -.005 .047 .075 -.041 .004 -.009 .007 .079 -.056 -.084 -.078 -.160 -.024 --.122
H
I -.104 -.175 a -.114 -.062 .036 -.042 -.113 -.065 -.102 -.012 -.062 -.013 -.040 .013
J
N
O
-.049 -.047 .090 -.129 -.141 -.100 -.007 -.048 -.072 -.070 -.061 -.016 .029 .004
Q3
Table I. Correlations Between CPQ Factors and Behavior Rating Scale Factors for Normal Students
-.115 -.124 .094 -.060 -.060 -.064 -.138 -.108 -.062 --.059 -.045 -.020 -.120 -.069
Q4
Ca
7.
B
.015 .256 a -.021 .010 -.099 .166 -.145 ,106 -.008 .027 .146 .066 -.072 ,037
A
.219 a .154 .097 -.115 -.271 a -.126 .160 ,044 -.030 .091 -.210 a -.167 .209 a -.106
ap < .01.
Q4
Q~
J N 0
H I
C D E F G
B
A
.065 .102 .135 .050 .033 ,212 a .233 a .144 -.195 -,061 .032 -.195 .117 ,007
C -.051 .111 -.124 -.010 .051 .051 -.069 -.149 .055 -.096 .058 .194 .037 .234 a
D .090 .166 .240 a .075 .116 .024 -.042 .188 -.122 .030 .058 -.102 .149 -.008
E
F -.131 -.072 -.109 .083 .092 .012 -.017 .038 .008 -.032 .107 .061 -.032 -.068
G -.035 .150 -.041 .119 .068 .272 a -.269 a .076 -.201 .120 .227 a -.033 -.003 .118
H
I -.024 -.110 -.031 -.093 .220 a .089 -.105 -.046 .168 -.013 -.014 .124 -.040 .159
J
N
0
Table II. Correlations Between CPQ Factors and Behavior Rating Scale Factors for Underachievers Q3 .106 .037 .099 -.089 -.167 -.186 .053 .093 .008 .010 -,182 -.153 .113 -.097
Q4 .018 -.108 -.044 -.266 a -.073 -.267 a .220 a -.125 .090 -.063 -.200 .115 .014 -.203
112
Willis and Seymour
Table II presents the correlations obtained on the underachiever group. As before, the items included from the behavior rating scale were those that had 100% agreement. Of 140 correlations, 16 were significant at the .01 level. Of these only 2, those for Factor A and Factor B, reflected correlations between the CPQ and behavior rating scale items assigned to them. When correlations were derived using all the items that had at least two-thirds agreement, there were 6 significant correlations, none of which reflected a relationship between a CPQ scale and its associated rating scale items. Data from the other referred group, students with behavior problems, provided no additional support for CPQ validity.
DISCUSSION
The data obtained in this study represent an attempt to deal with the criticism that the CPQ has not been shown to relate to behavior external to itself. Scores on the CPQ were correlated with ratings made by teachers using items judged to measure the same behaviors as the CPQ. The results strongly suggest that the CPQ does not measure the traits it claims to. This was true of data obtained from a normal sample of children as well as two referred groups. Without data that demonstrate that CPQ scores are associated with other measures of the target behavior, the use of such an instrument in a battery of psychological tests is questionable. The absence of a relationship between ratings and CPQ scores points out clearly the dangers of relying exclusively on a factor analytic approach in developing and defining scales. While a set of items may cluster together and may appear to contain a common thread o f meaning, there is no guarantee that the label the test developer attaches to the scale actually reflects what is being measured. In the case of the CPQ, the evidence presented here suggests that the labels do not reflect what is being measured.
REFERENCE
Anastasi, A. Review of IPAT Children's Personality Questionnaire. In O. Boros (Ed.), The sixth mental measurement yearbook. Highland Park, New Jersey: Gryphon Press, 1965. Cattell, R., & Porter, R. Handbook for the Children's Personality Questionnaire. Champaign, Illinois: Institute for Personality and Ability Testing, 1972. Giles, D. Item analysis and scale validation for the Wellington Child Behavior Rating Scale. Unpublished master's thesis, University of Western Ontario, 1976. Jackson, D. Review of IPAT edition of HSPQ. In O. Buros (Ed.), The seventh mental measurement handbook. Highland Park, New Jersey: Gryphon Press, 1972. Layton, W. Review of IPAT Children's Personality Questionnaire. In O. Buros (Ed.), The seventh mental measurement yearbook. Highland Park, New Jersey: Gryphon Press, 1965.
CPQ Validity
113
Lessing, E., & Barbera, L. Relationships among scores on the California Test of Personality, the IPAT Children's Personality Questionnaire and the IPAT High School Personality Questionnaire. Educational and Psychological Measurement, 1973, 33, 985-992. Lessing, E., & Zagorin, S. Correlation between Lorge-Thorndike IQ and Factor B of the IPAT Children's Personality Questionnaire. Psychological Reports, 1969, 24, 569-570. Spivack, G., & Swift, M. Devereux Elementary School Behavior Rating Scale. Devon, Pennsylvania: The Devereux Foundation, 1967.
