Journal ol Consulting and Clinkal fsycAology 1975, Vol. 43, No. 6, 851-857
Influence of Overlapping and Nonoverlapping Items on the Theoretical Interrelationships of MMPI Scales James A. Wakefield, Jr., Peggy E. Bradley, and Eugene B, Doughtie University of Houston Irvin A. Kraft Baylor College of Medicine The scales of the Minnesota Multiphasic Personality Inventory (MMPI) were scored (a) for items appearing only on one scale (nonoverlapping) and (b) for items appearing on more than one scale (overlapping). The correspondence of each of these two sets of scales to a theoretical structure composed of neuroticism, psychoticism, and extraversion dimensions was considered using a method presented by Wakefield and Doughtie. The correspondence of the scales composed of nonoverlapping items with the theoretical structure was insignificant. The correspondence of the scales composed of overlapping items conformed extremely closely to the theoretical structure. These results are interpreted as indicating that neuroticism, psychoticism, and extraversion factors found in the MMPI are due to items that appear on more than one scale. Considering the manner in which the scales were constructed, it is argued that the factors attributable to the overlapping items validly represent the similarity of the criteria against which the scales were validated.
Wakefield, Yom, Bradley, Doughtie, Cox, and Kraft (1974) have demonstrated that 9 of the 10 clinical scales of the Minnesota Multiphasic Personality Inventory conform to Eysenck's (1970) three-dimensional model of personality. The three theoretical dimensions are neuroticism, psychoticism, and extraversion. The Hy, D, and Hs scales of the MMPI were considered measures of neuroticism. The Pa, ,Pt, and Sc scales were considered measures of psychoticism. Si was considered a negative measure of extraversion. The Ma scale was considered to measure both neuroticism and extraversion, and Pd, both psychoticism and extraversion. The Mf scale was the only clinical scale that was not considered in the threedimensional framework. The fit of the MMPI scales to the theoretical arrangement stated above was tested by a method presented by Wakefield and Doughtie (1973). The distances between scales in the factor space were taken as measures of the dissimilarity of the scales. A nonp.arametric test was employed to determine whether the pattern of interscale distances in factor space
conformed to a pattern of dissimilarity derived from the theoretical arrangement of scales. This test was performed for both male and female adult subjects. The correspondence of the scales to the theoretical arrangement was significant in both cases, although it was considerably stronger for the females. It was concluded that the nine MMPI clinical scales measure personality in a fashion consistent with Eysenck's (1970) theory. Now a controversy, reviewed by Stein (1968), has arisen concerning the problem of overlapping items in the MMPI scales. This problem results from the manner by which the MMPI scales were developed—the criterion approach. This method consists of identifying items that reliably discriminate a criterion group (e.g., schizophrenics),from a comparison group (e.g., normals). In the case of the MMPI, the scales were developed independently but were composed of items from the same initial item pool. Unavoidably, a large number of items were scored for more than one scale. The controversy surrounding the overlapping items is concerned with the methodological status of the overlapping items. One group Requests for reprints should be sent to Eugene B. (Guilford, 1952; Shure & Rogers, 1965) has Doughtie, Department of Psychology, University of Houston, Houston, Texas 77004. viewed overlapping items as contributing arti851
852
WAKEFIELD, BRADLEY, DOUGHTIE, AND KRAFT
ficially to the relationships among the scales. This position disqualifies findings from factoranalytic studies of the MMPI. Shure and Rogers (1965) considered the "neurotic" and "psychotic" factors of the MMPI to be artifacts of the overlapping items. The opposite point of view (Anderson & Bashaw, 1966; Wheeler, Little, & Lehner, 1951) holds that the items common to more than one scale reflect common factors in the criteria used to develop the scales. As the items in each scale were independently validated against their criteria, the items included in more than one scale validly measure common aspects of more than one criterion. Stein (1968) mentioned two attempts to develop "pure" or nonoverlapping scales. The scales of Welsh (1956) eliminated only the overlapping items that were scored in the same direction. By retaining the items scored in opposite directions for two scales, the scales are not completely independent. However, Adams and Horn (1965) presented a set of completely independent scales.1 If the overlapping items do represent valid variances, they must represent variance that the scales containing them share (or in the case of oppositely scored items, variance that is shared negatively). If the overlapping items of two scales represent the shared variance, the unique variance of each scale should be represented by the nonoverlapping items. By virtue of their appearing on only one scale, these items must not measure the shared aspects of the criteria employed to select items for the scales. For example, items that are scored for both the Ey and D scales measure some more general factor or trait that includes the criteria for both scales—that is, neuroticism. Items that appear on only one of these scales do not measure neuroticism in general but rather a particular variation within the general trait. If the general traits neuroticism, psychoticism, and extraversion are validly measured by the overlapping items and more specific traits measured by nonoverlapping items, an analysis such as that of Wakefield et*al. (1974) of scales scored for the two types of items 1 Both the Welsh and the Adams and Horn scales can also be found in the MMPI handbook (Dahlstrom, Welsh, & Dahlstrom, 1972).
should yield a certain pattern. That is, the scales scored for overlapping items only should be highly related to the theoretical model of the interrelations of scales, whereas the scales scored for the nonoverlapping items only should be unrelated to the theoretical model. METHOD Subjects The subjects were 100 adults who took the MMPI in the course of obtaining psychiatric services for their children. Scales The 10 clinical scales and the 3 validity scales of the MMPI were scored twice. The two scorings produced a measure from the overlapping items and a measure from the nonoverlapping items of each scale. The nonoverlapping MMPI scales were obtained by eliminating all items scored, in either direction, for more than one scale. These scales correspond to those of Adams and Horn (1965). Similarly, the overlapping MMPI scales were obtained by eliminating all items scored for only one scale. Thus, for example, the overlapping Sc scale consisted of that set of items scored for the Sc scale and at least one other clinical or validity scale.
Factor Analysis The 3 validity scales and 10 clinical scales were scored for overlapping and nonoverlapping measures for each subject. A correlation matrix of the 26 variables was obtained and tested for significance using Bartlett's test (Weiss, 1970). The resulting chi-square of 1,535.26 (df = 325, p < .001) indicated that the matrix was significantly different from the identity matrix and thus suitable for factoring. All principal factors (Harman, 1970) with positive eigenvalues were extracted from the correlation matrix and rotated by varimax.
Comparison of Distances in Factor Space Between Scales As in the Wakefield et al. (1974) study, distances between scales were computed and compared for relative magnitude. There were 176 comparisons of two distances derived from the theoretical arrangement of MMPI scales according to Eysenck's (1970) theory. At random, one half of the distance comparisons (88) would be expected to be ordered in the theoretically correct direction. The normal approximation to a binomial test for goodness of fit (Siegel, 1956, pp. 36-42) was used to compute the probability that the observed number of correct comparisons could have occurred at random.2 1 Since the theoretical relationship between the MMPI scales and Eysenck's personality dimensions and the empirical method used to test this relationship have been recently presented (Wakefield et al., 1974),' this material is not repeated here,
853
OVERLAPPING AND NONOVERLAPPING ITEMS ON THE MMPI TABLE 1 CORRELATIONS OF MMPI NONOVERLAPPING AND OVERLAPPING SCALES roa 100 ADULTS Scale
L
F
K
Hs
D
Hy
Pd
Mi
Pa
PI
Sc
Ma
Si
Nonoverlapping scales
Nonoverlapping L F K Hs D Hy Pd Mf
Pa Pt
Sc Ma Si Overlapping L P K Hs D Hy Pd Mf Pa Pt Sc Ma Si
-.062 -.034 -.111 -.189 .079 -.121 -.009 -.204 -.121 -.184 -.086 .011
-.015 .161 .160 -.100 .428 -.092 .229 .297 .511 .200 .274
-.364 -.202 .155 -.325 .071 -.075 -.384 -.313 -.291 -.059
.210 .010 .233 .149 .299 .299 .452 .134 .137
.045 .342 .163 .255 .207 .391 .138 .167
-.095 .212 .208 .052 -.162 -.085 -.111
-.111 .128 .248 .613 .157 .261
.240 .246 .023 -.109 -.073
.423 .337 .030 .183
.382 .129 .281
.159 .366
.331 -.004 .250 -.140 -.012 -.014 -.069 -.173 .004 -.139 -.084 -.228 -.066
-.286 .371 -.341 .348 .207 .190 .441 .085 ,229 .510 .546 .417 .281
.090 -.058 .397 -.217 -.184 -.029 -.210 -.087 -.051 -.317 -.270 -.275 -.209
-.253 .352 -.254 .634 .435 .445 .360 .202 .225 .447 .458 .256 .256
-.225 .316 -.377 .404 .426 .267 .265 .154 .272 .485 .484 .245 .418
.147 -.088 .226 .202 .051 .313 -.080 .143 .135 .007 -.041 .087 -.122
-.504 .344 -.504 .460 .398 .279 .463 -.010 .261 .594 .559 .290 .370
.088 .105 -.006 .165 .179 .261 .109 .441 .207 .184 .135 .190 .008
-.351 .216 -.237 .345 .383 .264 .280 .142 .399 .414 .436 .180 .182
-.304 .252 -.488 .422 .356 .245 .292 .339 .214 .510 .459 .353 .335
-.331 .561 -.528 .595 .516 .361 .618 .286 .493 .732 .792 .507 .492
-.068 .076 -.253 .127 .062 .010 .214 .075 .066 .185 .112 .265 .033
.537 .572 .273 .263
.890 .445 .701
.476 .635
.121
-.104 -.204 .309 -.294 .283 .384 .018 .156 .051 .225 .435 .446 .125 .579
Overlapping scales Overlapping
L F K Hs D
Hy Pd Mf Pa PI Sc Ma Si
-.224 .406 -.259 -.122 -.100 -.294 .139 -.142 -.363 -.369 -.204 -.168
-.269 .432 .552 .390 .644 .141 .530 .610 .720 .245 .441
.RESULTS
-.310 -.246 .098 -.249 -.139 -.178 -.567 -.491 -.305 -.529
.638 .704 .428 .329 .391 .655 .655 .380 .395
.486 .502 .309 .496 .754 .666 .156 .677
.474 .303 .400 .442 .433 .214 .051
.209 .525 .625 .590 .447 .266
.177 .309 .257 .302 .239
retical order. Using the normal approximation Table 1. contains the intercorrelations of the to a binomial test, a z score of .68 was comoverlapping and nonoverlapping MMPI scales. puted. This result is not significantly different Table 2 contains the 15 rotated principal from chance. For the overlapping scales, howfactors extracted from this matrix. Tables 3 ever, 141 of the 176 distance comparisons were and 4 contain the distances in 15-dimensional in the correct order, yielding a z score of 7.92 space between each pair of nonoverlapping (p < .001). scales and the distances between each pair of DISCUSSION overlapping scales. A list of correct and incorrect distance comparisons for the nonoverThe results of the present study clearly lapping scales and a list of correct and in- show that the relationship between the MMPI correct distance comparisons for the over- and the three personality dimensions of Eyslapping scales are presented in Tables 5 and 6. enck (1970)—neuroticism,"psychoticism, and For the nonoverlapping scales, 93 of the 176 extraversion—is accounted for by the items distance comparisons were in the correct theo- that are scored on more than one scale. The
TABLE 2 VARIMAX ROTATED PRINCIPAL FACTORS ACCOUNTING TOR 65.4% OF TOTAL VARIANCE Factor
Nonoverlapping L F K Us D Hy Pd if/ Pa PI Sc Ma Si Overlapping L F K Hs D Hy Pd
Mf Pa Pt
Sc Ma Si
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
.024 .284 .040 .191 .238 -.025 .285 .120 .241 .055 .482 .106 .110
-.012 .205 -.131 .133 .311 -.094 .276 -.046 .141 .287 .353 -.046 .620
-.043 .061 -.217 .685 .123 .087 .177 .063 .140 .178 .293 .023 .052
-.087 -.080 -.039 .100 .116 .172 -.162 .629 .111 .344 .095 -.035 -.071
-.048 .182 -.599 .206 .185 -.105 .205 -.103 .008 .266 .192 .520 -.131
-.058 .563 -.075 .014 -.034 -.009 .213 -.072 .048 .206 .365 .103 .194
-.082 .107 -.122 .167 .044 .100 .003 .149 .651 .438 .087 -.034 .132
-.041 .146 -.180 .008 .167 -.040 .633 -.025 -.007 .164 .238 -.011 .054
.599 -.059 -.139 -.077 -.266 .081 -.141 .001 -.204 -.039 -.177 -.111 .023
.050 -.040 .171 -.070 .125 .622 -.024 .144 .187 -.002 -.156 -.007 -.074
-.028 -.035 -.053 .011 .361 .025 .077 .092 .044 -.077 .064 .007 -.064
.000 -.007 -.002 -.043 -.014 .010 .009 -.039 .017 -.078 -.045 .004 -.041
-.021 -.020 -.044 -.012 .009 -.001 .047 -.064 .049 -.110 .287 -.013 .023
-.006 .098 .124 -.107 -.002 -.007 .001 -.065 .003 .002 -.049 .060 -.080
.003 .001 -.027 -.055 -.002 -.004 .001 -.038 -.038 .153 .001 -.017 .081
^ X
H2 J3
w >o t=i -
0 o o K H
-.150 .731 -.065 .267 .470 .412 .733 .086 .636 .487 .545 .241 .181
-.097 .301 -.436 .320 .602 -.053 .070 .152 .176 .577 .504 .028 .876
-.134 .208 .007 .687 .350 .596 .193 .149 .086 .238 .289 .097 .076
.224 .041 -.080 .150 .173 .228 .103 .651 .119 .194 .106 .265 .085
-.007 -.018 -.422 .113 .084 -.082 .210 .097 .006 .234 .097 .326 .101
-.147 .139 -.126 .194 -.138 .087 .220 .102 .062 .246 .381 .505 .015
-.352 .019 -.203 .070 .120 .022 .064 -.033 .210 .184 .210 .024 .010
-.382 .008 -.313 .168 .080 .170 .180 -.080 .039 .267 .157 .072 .066
.433 .017 .274 -.135 .030 .028 -.070 -.109 .025 -.095 -.037 -.220 -.052
.119 -.122 .213 .270 .111 .344 -.075 .075 .175 .060 -.026 .104 -.097
-.019 .092 -.092 .055 .049 .046 -.087 -.044 .058 .094 .261 .025 .090
.068 -.117 .319 -.019 .231 .152 .128 .047 -.029 .028 -.066 -.014 -.015
.168 -.133 -.017 .051 .031 -.019 -.045 .090 .143 -.049 .022 .034 -.005
.062 .042 -.005 .073 .093 .162 -.014 .064 -.050 .044 -.024 -.227 .039
-.015 .041 -.010 .058 -.036 .078 .015 .029 -.026 -.152 -.070 .016 -.036
5 > o W »
S H
855
OVERLAPPING AND NONOVERLAPPING ITEMS ON THE MMPI TABLE 3 DISTANCES BETWEEN NONOVERLAPPING MMPI SCALES IN IS-DIMENSIONAL SPACE Scale 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.
L F K Hs D Hy Pd iff Pa Pt Sc Ma Si
1
2
3
4
5
6
7
8
9
10
11
12
1.043 .975 1.112 1.083 .855 1.172 .990 1.171 1.119 1.259 .938 .951
1.216 .900 .800 1.080 .654 1.092 .871 .739 .590 .690 .703
1.350 1.157 .859 1.371 .980 1.146 1.354 1.418 1.196 1.094
.769 1.041 .920 ,933 .822 .759 .766 .824 .927
.907 .690 .822 .757 .771 .712 .712 .787
1.196 .700 .853 .994 1.254 .962 1.073
1.212 1.020 .901 .627 .846 .839
.818 .799 1.108 .978 1.051
.653 .907 .937 .870
.817 .819 .777
.882 .778
.984
items that were scored for only one scale showed almost no relationship to Eysenck's theoretical dimensions. This study concurs with the conclusion of Shure and Rogers (1965) "that the overlap items are responsible for the presence of the N-Fjieurotic] and P-type [psychotic] factors found in the empirical studies" (p. 17). However, their further comments that call into question the validity of the neurotic and psychotic personality dimensions are not supported. Since the neurotic and psychotic personality dimensions antedate the MMPI, no methodological criticism of the MMPI alone can disqualify those dimensions. A number of instruments other than the MMPI have been constructed that attempt to measure those
13
and other personality dimensions. In fact, in an early review of the MMPI, Eysenck (1949) suggested that the MMPI was simply a questionnaire of the familiar "neuroticismcum-extraversion" type. The present study supports the view of Anderson and Bashaw (1966). They held that the number of items common to two scales could be considered an index of the similarity between the two latent variables measured by the scales. Items that measure a characteristic common to closely associated latent variables should be selected for scales that measure the two variables. Extending their argument somewhat, it may be said that items measuring aspects unique to one latent variable should be selected only for the scale measuring that
TABLE 4 DISTANCES BETWEEN OVERLAPPING MMPI SCALES IN IS-DIMENSIONAL SPACE Scale 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.
L F K Hs D Hy Pd Mf Pa Pt Sc Ma Si
1
2
3
4
5
6
7
8
9
10
11
12
1.352 .759 1.420 1.330 1.278 1.397 .975 1.219 1.511 1.518 1.282 1.349
1.428 .854 .715 .876 .511 .994 .525 .663 .549 .926 .850
1.488 1.473 1.192 1.426 1.260 1.288 1.678 1.649 1.424 1.627
.694 .548 .849 .858 .806 .681 .704 .853 .947
.820 .795 .892 .665 .553 .692 1.078 .628
.801 .871 .761 .935 .941 .970 1.231
.942 .544 .666 .684 .725 1.047
.857 .915 .994 .736 .978
.666 .621 .803 .928
.365 .811 .637
.805 .736
1.092
13
WAKEFIELD, BRADLEY, DOUGHTIE, AND KRAFT
856
TABLE S 176 DISTANCE COMPARISONS FOR NONOVERLAPPING SCALES Long distances Middle distances Si-Hs Si-D Si-IIy Si-Pa. Si-Pt Si-Sc Sc-Hs Sc-D Sc-Hy Pt-Hs Pt-D Pt-Hy Par-Hs Pa-D Pa-Hy Ma-IIs Ma-D Ma-Hy Sc-Pd Pi-Pi Pa-Pd Ma-Pd
Short distances
Si-Ma
Si-Pd
Ily-D
D-Hs
Hy-Hs
Sc-Pt
Pt-Pa
Sc-Pa
1
0 1 0 0 1 1 1 1 0
1 0 1 0 0 0 0
1 1 1 1 1 1 0 0
0 0
1 0
1 0 0 0 0 0
0 0 0 0
1 1 1 1 1 1 1' 1
1 0 1 0 0 0 0 0
0 0
0
0
0 0
1 1
0 0
0
1
0
1
0
0 0 0 0
0
1 1 1 1
0 0 0 0
0 0
0
0 0 0 0 0 0 0 0 0
0 1
0 0
1
0
1
0 1 1 1 1 1 0 1 1 0
1 1
1 1 1 1 1 1
0
1
1 1
0
1
1
0
1 1
0
1
0 0 0
0
1
1 1 1
1
1
1 1
0
1
1
1
1 1
0
1 1
1
1
0
1 1
0
0
1
1 1
0
0
1 1 1
1
1
1 1
1
1
1
1
• .68, ns.
Note. 1 = correct; 0 = incorrect; number correct
TABLE 6 176 DISTANCE COMPARISONS TOR OVERLAPPING SCALES Long distances Middle distances
Si-Ma
Si-Pd
Si-Hs Si-D Si-Hy Si-Pa Si-Pt Si-Sc Sc-Hs Sc-D Sc-Hy Pt-Hs Pt-D Pt-Hy Pa-Hs Pa-D Pa-Hy Ma-Hs Ma-D Ma-Hy Sc-Pd Pt-Pd Pa-Pd Ma-Pd
1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1
0
1 1 1 1 1
Short distances
Hy-D
D-Hs
1 0 1 1 0 0 0 0 1 0 0 1 0 0 0 1 1 1 0 0 0 0
1 0 1 1
0
1 1
0
1
0 0
1 1
0
1 1 1 1
0
0 0
1
Note. 1 = correct; 0 = incorrect; number correct = 141; z - 7.92, p < .001.
Hy-Hs
Sc-Pt
Pt-Pa,
Sc-Pa
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 I 1
1 0 1 1 0 1 1 1 1 1 0
1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0
1 1
0
1 1 1 1 1
0 0
1
1
OVERLAPPING AND NONOVERLAPPING ITEMS ON THE MMPI latent variable. Thus, theoretical interrelationships among latent variables should be observable only in items occurring on more than one scale. No theoretical interrelationship should be observable in the items occurring on only one scale. The results of the present study conformed to the two previous statements. They showed an extremely strong pattern of theoretical interrelationships for the overlapping items and no significant pattern of interrelationships for thenonoverlapping items. An early criticism of the MMPI (Benton, 1949) was that it was inadequate for differentiating between different categories of abnormality even though it did discriminate between normals and abnormals and indicate severity of abnormality. The present results suggest that the mixture in each scale of. (a) overlapping items measuring global personality dimensions and (b) nonoverlapping items possibly measuring finer qualities of particular categories of abnormality may impede the differential diagnosis of personality with the MMPI. The distinction between the two types of items contained in the MMPI scales and the different theoretical levels of personality each type measures leads to a possible method of improving the diagnostic value of the MMPI. A hierarchical approach to personality measurement consistent with Eysenck's (1947) theory of personality is suggested. The first stage in this approach would rely on scales scored only for overlapping items to determine the global personality dimension -(neuroticism or psychoticism) that best characterized an individual. The second stage would rely on scales scored for nonoverlapping items to make finer distinctions within neuroticism or within psychoticism. Although the present results are consistent with this two-stage process of personality diagnosis, a great deal of research with carefully chosen criterion groups is necessary to add empirical substance to this theoretically derived technique for improving differential personality diagnosis.
857
REFERENCES Adams, D. K., & Horn, J, L. Nonoverlapping keys for the MMPI scales. Journal of Consulting Psychology, 1965, 29, 284. Anderson, H. E., & Bashaw, W. L. Further comments on. the internal structure of the MMPI. Psychological Bulletin, 1966, 66, 211-213. Benton, A. L. Review of the MMPI. In 0. K. Euros (Ed.), Third mental measurements yearbook. New Brunswick, N J.: Rutgers University Press, 1949. Dahlstrom, W. G., Welsh, G. S., & Dahlstrom, L. E. MMPI handbook, Clinical interpretation (Vol. 1) (Rev. ed.). Minneapolis: University of Minnesota Press, 1972. Eysenck, H. J. Dimensions of personality. London: Kegan, Paul, 1947. Eysenck, H. J. Review of the MMPI. In O. K. Euros (Ed.), Third mental measurements yearbook. New Brunswick, N.J.: Rutgers University Press, 1949. Eysenck, H. J. A dimensional system of psychodiagnostics. In A. R. Mahrer (Ed.), New approaches to personality classification. New York: Columbia University Press, 1970. Guilford, J. P. When not to factor analyze. Psychological Bulletin, 19S2, 49, 26-37. Harman, H. H. Modern factor analysis. Chicago: University of Chicago Press, 1970. Shure, G. H., & Rogers, M. S. Note of caution on the factor analysis of the MMPI. Psychological Bulletin, 1965, 63, 14-18. Siegel, S. Nonparametric statistics. New York: McGrawHill, 1956. Stein, K. B. The TSC scales: The outcome of a cluster analysis of the 550 MMPI items. In P. McReynolds (Ed.), Advances in psychological assessment (Vol. 1). Palo Alto, Calif.: Science and Behavior Books, 1968. Wakefield, J. A., Jr., & Doughtie, E, B. The geometric relationship between Holland's personality typology and the Vocational Preference Inventory. Journal of Counseling Psychology, 1973, 20, 513-518. Wakefield, J. A., Jr., Yom, B. L., Bradley, P. E., Doughtie, E. B., Cox, J. A., & Kraft, I. A. Eysenck's personality dimensions: Model for the MMPI. British Journal of Social and Clinical Psychology, 1974, 13, 413-420. Weiss, D. J. Factor analysis and counseling research. Journal of Counseling Psychology, 1970, 17, 477-485. Welsh, G. S. Factor dimensions A and R. In G. S. Welsh & W. G. Dahlstrom (Eds.), Basic readings on the MMPI in psychology and medicine. Minneapolis: University of Minnesota Press, 1956. Wheeler, W. M., Little, K. B., & Lehner, G. F. J. The internal structure of the MMPI. Journal of Consulting Psychology, 1951,15, 134-141. (Received March 31,1975)
Influence of Overlapping and Nonoverlapping Items on the Theoretical Interrelationships of MMPI Scales James A. Wakefield, Jr., Peggy E. Bradley, and Eugene B, Doughtie University of Houston Irvin A. Kraft Baylor College of Medicine The scales of the Minnesota Multiphasic Personality Inventory (MMPI) were scored (a) for items appearing only on one scale (nonoverlapping) and (b) for items appearing on more than one scale (overlapping). The correspondence of each of these two sets of scales to a theoretical structure composed of neuroticism, psychoticism, and extraversion dimensions was considered using a method presented by Wakefield and Doughtie. The correspondence of the scales composed of nonoverlapping items with the theoretical structure was insignificant. The correspondence of the scales composed of overlapping items conformed extremely closely to the theoretical structure. These results are interpreted as indicating that neuroticism, psychoticism, and extraversion factors found in the MMPI are due to items that appear on more than one scale. Considering the manner in which the scales were constructed, it is argued that the factors attributable to the overlapping items validly represent the similarity of the criteria against which the scales were validated.
Wakefield, Yom, Bradley, Doughtie, Cox, and Kraft (1974) have demonstrated that 9 of the 10 clinical scales of the Minnesota Multiphasic Personality Inventory conform to Eysenck's (1970) three-dimensional model of personality. The three theoretical dimensions are neuroticism, psychoticism, and extraversion. The Hy, D, and Hs scales of the MMPI were considered measures of neuroticism. The Pa, ,Pt, and Sc scales were considered measures of psychoticism. Si was considered a negative measure of extraversion. The Ma scale was considered to measure both neuroticism and extraversion, and Pd, both psychoticism and extraversion. The Mf scale was the only clinical scale that was not considered in the threedimensional framework. The fit of the MMPI scales to the theoretical arrangement stated above was tested by a method presented by Wakefield and Doughtie (1973). The distances between scales in the factor space were taken as measures of the dissimilarity of the scales. A nonp.arametric test was employed to determine whether the pattern of interscale distances in factor space
conformed to a pattern of dissimilarity derived from the theoretical arrangement of scales. This test was performed for both male and female adult subjects. The correspondence of the scales to the theoretical arrangement was significant in both cases, although it was considerably stronger for the females. It was concluded that the nine MMPI clinical scales measure personality in a fashion consistent with Eysenck's (1970) theory. Now a controversy, reviewed by Stein (1968), has arisen concerning the problem of overlapping items in the MMPI scales. This problem results from the manner by which the MMPI scales were developed—the criterion approach. This method consists of identifying items that reliably discriminate a criterion group (e.g., schizophrenics),from a comparison group (e.g., normals). In the case of the MMPI, the scales were developed independently but were composed of items from the same initial item pool. Unavoidably, a large number of items were scored for more than one scale. The controversy surrounding the overlapping items is concerned with the methodological status of the overlapping items. One group Requests for reprints should be sent to Eugene B. (Guilford, 1952; Shure & Rogers, 1965) has Doughtie, Department of Psychology, University of Houston, Houston, Texas 77004. viewed overlapping items as contributing arti851
852
WAKEFIELD, BRADLEY, DOUGHTIE, AND KRAFT
ficially to the relationships among the scales. This position disqualifies findings from factoranalytic studies of the MMPI. Shure and Rogers (1965) considered the "neurotic" and "psychotic" factors of the MMPI to be artifacts of the overlapping items. The opposite point of view (Anderson & Bashaw, 1966; Wheeler, Little, & Lehner, 1951) holds that the items common to more than one scale reflect common factors in the criteria used to develop the scales. As the items in each scale were independently validated against their criteria, the items included in more than one scale validly measure common aspects of more than one criterion. Stein (1968) mentioned two attempts to develop "pure" or nonoverlapping scales. The scales of Welsh (1956) eliminated only the overlapping items that were scored in the same direction. By retaining the items scored in opposite directions for two scales, the scales are not completely independent. However, Adams and Horn (1965) presented a set of completely independent scales.1 If the overlapping items do represent valid variances, they must represent variance that the scales containing them share (or in the case of oppositely scored items, variance that is shared negatively). If the overlapping items of two scales represent the shared variance, the unique variance of each scale should be represented by the nonoverlapping items. By virtue of their appearing on only one scale, these items must not measure the shared aspects of the criteria employed to select items for the scales. For example, items that are scored for both the Ey and D scales measure some more general factor or trait that includes the criteria for both scales—that is, neuroticism. Items that appear on only one of these scales do not measure neuroticism in general but rather a particular variation within the general trait. If the general traits neuroticism, psychoticism, and extraversion are validly measured by the overlapping items and more specific traits measured by nonoverlapping items, an analysis such as that of Wakefield et*al. (1974) of scales scored for the two types of items 1 Both the Welsh and the Adams and Horn scales can also be found in the MMPI handbook (Dahlstrom, Welsh, & Dahlstrom, 1972).
should yield a certain pattern. That is, the scales scored for overlapping items only should be highly related to the theoretical model of the interrelations of scales, whereas the scales scored for the nonoverlapping items only should be unrelated to the theoretical model. METHOD Subjects The subjects were 100 adults who took the MMPI in the course of obtaining psychiatric services for their children. Scales The 10 clinical scales and the 3 validity scales of the MMPI were scored twice. The two scorings produced a measure from the overlapping items and a measure from the nonoverlapping items of each scale. The nonoverlapping MMPI scales were obtained by eliminating all items scored, in either direction, for more than one scale. These scales correspond to those of Adams and Horn (1965). Similarly, the overlapping MMPI scales were obtained by eliminating all items scored for only one scale. Thus, for example, the overlapping Sc scale consisted of that set of items scored for the Sc scale and at least one other clinical or validity scale.
Factor Analysis The 3 validity scales and 10 clinical scales were scored for overlapping and nonoverlapping measures for each subject. A correlation matrix of the 26 variables was obtained and tested for significance using Bartlett's test (Weiss, 1970). The resulting chi-square of 1,535.26 (df = 325, p < .001) indicated that the matrix was significantly different from the identity matrix and thus suitable for factoring. All principal factors (Harman, 1970) with positive eigenvalues were extracted from the correlation matrix and rotated by varimax.
Comparison of Distances in Factor Space Between Scales As in the Wakefield et al. (1974) study, distances between scales were computed and compared for relative magnitude. There were 176 comparisons of two distances derived from the theoretical arrangement of MMPI scales according to Eysenck's (1970) theory. At random, one half of the distance comparisons (88) would be expected to be ordered in the theoretically correct direction. The normal approximation to a binomial test for goodness of fit (Siegel, 1956, pp. 36-42) was used to compute the probability that the observed number of correct comparisons could have occurred at random.2 1 Since the theoretical relationship between the MMPI scales and Eysenck's personality dimensions and the empirical method used to test this relationship have been recently presented (Wakefield et al., 1974),' this material is not repeated here,
853
OVERLAPPING AND NONOVERLAPPING ITEMS ON THE MMPI TABLE 1 CORRELATIONS OF MMPI NONOVERLAPPING AND OVERLAPPING SCALES roa 100 ADULTS Scale
L
F
K
Hs
D
Hy
Pd
Mi
Pa
PI
Sc
Ma
Si
Nonoverlapping scales
Nonoverlapping L F K Hs D Hy Pd Mf
Pa Pt
Sc Ma Si Overlapping L P K Hs D Hy Pd Mf Pa Pt Sc Ma Si
-.062 -.034 -.111 -.189 .079 -.121 -.009 -.204 -.121 -.184 -.086 .011
-.015 .161 .160 -.100 .428 -.092 .229 .297 .511 .200 .274
-.364 -.202 .155 -.325 .071 -.075 -.384 -.313 -.291 -.059
.210 .010 .233 .149 .299 .299 .452 .134 .137
.045 .342 .163 .255 .207 .391 .138 .167
-.095 .212 .208 .052 -.162 -.085 -.111
-.111 .128 .248 .613 .157 .261
.240 .246 .023 -.109 -.073
.423 .337 .030 .183
.382 .129 .281
.159 .366
.331 -.004 .250 -.140 -.012 -.014 -.069 -.173 .004 -.139 -.084 -.228 -.066
-.286 .371 -.341 .348 .207 .190 .441 .085 ,229 .510 .546 .417 .281
.090 -.058 .397 -.217 -.184 -.029 -.210 -.087 -.051 -.317 -.270 -.275 -.209
-.253 .352 -.254 .634 .435 .445 .360 .202 .225 .447 .458 .256 .256
-.225 .316 -.377 .404 .426 .267 .265 .154 .272 .485 .484 .245 .418
.147 -.088 .226 .202 .051 .313 -.080 .143 .135 .007 -.041 .087 -.122
-.504 .344 -.504 .460 .398 .279 .463 -.010 .261 .594 .559 .290 .370
.088 .105 -.006 .165 .179 .261 .109 .441 .207 .184 .135 .190 .008
-.351 .216 -.237 .345 .383 .264 .280 .142 .399 .414 .436 .180 .182
-.304 .252 -.488 .422 .356 .245 .292 .339 .214 .510 .459 .353 .335
-.331 .561 -.528 .595 .516 .361 .618 .286 .493 .732 .792 .507 .492
-.068 .076 -.253 .127 .062 .010 .214 .075 .066 .185 .112 .265 .033
.537 .572 .273 .263
.890 .445 .701
.476 .635
.121
-.104 -.204 .309 -.294 .283 .384 .018 .156 .051 .225 .435 .446 .125 .579
Overlapping scales Overlapping
L F K Hs D
Hy Pd Mf Pa PI Sc Ma Si
-.224 .406 -.259 -.122 -.100 -.294 .139 -.142 -.363 -.369 -.204 -.168
-.269 .432 .552 .390 .644 .141 .530 .610 .720 .245 .441
.RESULTS
-.310 -.246 .098 -.249 -.139 -.178 -.567 -.491 -.305 -.529
.638 .704 .428 .329 .391 .655 .655 .380 .395
.486 .502 .309 .496 .754 .666 .156 .677
.474 .303 .400 .442 .433 .214 .051
.209 .525 .625 .590 .447 .266
.177 .309 .257 .302 .239
retical order. Using the normal approximation Table 1. contains the intercorrelations of the to a binomial test, a z score of .68 was comoverlapping and nonoverlapping MMPI scales. puted. This result is not significantly different Table 2 contains the 15 rotated principal from chance. For the overlapping scales, howfactors extracted from this matrix. Tables 3 ever, 141 of the 176 distance comparisons were and 4 contain the distances in 15-dimensional in the correct order, yielding a z score of 7.92 space between each pair of nonoverlapping (p < .001). scales and the distances between each pair of DISCUSSION overlapping scales. A list of correct and incorrect distance comparisons for the nonoverThe results of the present study clearly lapping scales and a list of correct and in- show that the relationship between the MMPI correct distance comparisons for the over- and the three personality dimensions of Eyslapping scales are presented in Tables 5 and 6. enck (1970)—neuroticism,"psychoticism, and For the nonoverlapping scales, 93 of the 176 extraversion—is accounted for by the items distance comparisons were in the correct theo- that are scored on more than one scale. The
TABLE 2 VARIMAX ROTATED PRINCIPAL FACTORS ACCOUNTING TOR 65.4% OF TOTAL VARIANCE Factor
Nonoverlapping L F K Us D Hy Pd if/ Pa PI Sc Ma Si Overlapping L F K Hs D Hy Pd
Mf Pa Pt
Sc Ma Si
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
.024 .284 .040 .191 .238 -.025 .285 .120 .241 .055 .482 .106 .110
-.012 .205 -.131 .133 .311 -.094 .276 -.046 .141 .287 .353 -.046 .620
-.043 .061 -.217 .685 .123 .087 .177 .063 .140 .178 .293 .023 .052
-.087 -.080 -.039 .100 .116 .172 -.162 .629 .111 .344 .095 -.035 -.071
-.048 .182 -.599 .206 .185 -.105 .205 -.103 .008 .266 .192 .520 -.131
-.058 .563 -.075 .014 -.034 -.009 .213 -.072 .048 .206 .365 .103 .194
-.082 .107 -.122 .167 .044 .100 .003 .149 .651 .438 .087 -.034 .132
-.041 .146 -.180 .008 .167 -.040 .633 -.025 -.007 .164 .238 -.011 .054
.599 -.059 -.139 -.077 -.266 .081 -.141 .001 -.204 -.039 -.177 -.111 .023
.050 -.040 .171 -.070 .125 .622 -.024 .144 .187 -.002 -.156 -.007 -.074
-.028 -.035 -.053 .011 .361 .025 .077 .092 .044 -.077 .064 .007 -.064
.000 -.007 -.002 -.043 -.014 .010 .009 -.039 .017 -.078 -.045 .004 -.041
-.021 -.020 -.044 -.012 .009 -.001 .047 -.064 .049 -.110 .287 -.013 .023
-.006 .098 .124 -.107 -.002 -.007 .001 -.065 .003 .002 -.049 .060 -.080
.003 .001 -.027 -.055 -.002 -.004 .001 -.038 -.038 .153 .001 -.017 .081
^ X
H2 J3
w >o t=i -
0 o o K H
-.150 .731 -.065 .267 .470 .412 .733 .086 .636 .487 .545 .241 .181
-.097 .301 -.436 .320 .602 -.053 .070 .152 .176 .577 .504 .028 .876
-.134 .208 .007 .687 .350 .596 .193 .149 .086 .238 .289 .097 .076
.224 .041 -.080 .150 .173 .228 .103 .651 .119 .194 .106 .265 .085
-.007 -.018 -.422 .113 .084 -.082 .210 .097 .006 .234 .097 .326 .101
-.147 .139 -.126 .194 -.138 .087 .220 .102 .062 .246 .381 .505 .015
-.352 .019 -.203 .070 .120 .022 .064 -.033 .210 .184 .210 .024 .010
-.382 .008 -.313 .168 .080 .170 .180 -.080 .039 .267 .157 .072 .066
.433 .017 .274 -.135 .030 .028 -.070 -.109 .025 -.095 -.037 -.220 -.052
.119 -.122 .213 .270 .111 .344 -.075 .075 .175 .060 -.026 .104 -.097
-.019 .092 -.092 .055 .049 .046 -.087 -.044 .058 .094 .261 .025 .090
.068 -.117 .319 -.019 .231 .152 .128 .047 -.029 .028 -.066 -.014 -.015
.168 -.133 -.017 .051 .031 -.019 -.045 .090 .143 -.049 .022 .034 -.005
.062 .042 -.005 .073 .093 .162 -.014 .064 -.050 .044 -.024 -.227 .039
-.015 .041 -.010 .058 -.036 .078 .015 .029 -.026 -.152 -.070 .016 -.036
5 > o W »
S H
855
OVERLAPPING AND NONOVERLAPPING ITEMS ON THE MMPI TABLE 3 DISTANCES BETWEEN NONOVERLAPPING MMPI SCALES IN IS-DIMENSIONAL SPACE Scale 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.
L F K Hs D Hy Pd iff Pa Pt Sc Ma Si
1
2
3
4
5
6
7
8
9
10
11
12
1.043 .975 1.112 1.083 .855 1.172 .990 1.171 1.119 1.259 .938 .951
1.216 .900 .800 1.080 .654 1.092 .871 .739 .590 .690 .703
1.350 1.157 .859 1.371 .980 1.146 1.354 1.418 1.196 1.094
.769 1.041 .920 ,933 .822 .759 .766 .824 .927
.907 .690 .822 .757 .771 .712 .712 .787
1.196 .700 .853 .994 1.254 .962 1.073
1.212 1.020 .901 .627 .846 .839
.818 .799 1.108 .978 1.051
.653 .907 .937 .870
.817 .819 .777
.882 .778
.984
items that were scored for only one scale showed almost no relationship to Eysenck's theoretical dimensions. This study concurs with the conclusion of Shure and Rogers (1965) "that the overlap items are responsible for the presence of the N-Fjieurotic] and P-type [psychotic] factors found in the empirical studies" (p. 17). However, their further comments that call into question the validity of the neurotic and psychotic personality dimensions are not supported. Since the neurotic and psychotic personality dimensions antedate the MMPI, no methodological criticism of the MMPI alone can disqualify those dimensions. A number of instruments other than the MMPI have been constructed that attempt to measure those
13
and other personality dimensions. In fact, in an early review of the MMPI, Eysenck (1949) suggested that the MMPI was simply a questionnaire of the familiar "neuroticismcum-extraversion" type. The present study supports the view of Anderson and Bashaw (1966). They held that the number of items common to two scales could be considered an index of the similarity between the two latent variables measured by the scales. Items that measure a characteristic common to closely associated latent variables should be selected for scales that measure the two variables. Extending their argument somewhat, it may be said that items measuring aspects unique to one latent variable should be selected only for the scale measuring that
TABLE 4 DISTANCES BETWEEN OVERLAPPING MMPI SCALES IN IS-DIMENSIONAL SPACE Scale 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.
L F K Hs D Hy Pd Mf Pa Pt Sc Ma Si
1
2
3
4
5
6
7
8
9
10
11
12
1.352 .759 1.420 1.330 1.278 1.397 .975 1.219 1.511 1.518 1.282 1.349
1.428 .854 .715 .876 .511 .994 .525 .663 .549 .926 .850
1.488 1.473 1.192 1.426 1.260 1.288 1.678 1.649 1.424 1.627
.694 .548 .849 .858 .806 .681 .704 .853 .947
.820 .795 .892 .665 .553 .692 1.078 .628
.801 .871 .761 .935 .941 .970 1.231
.942 .544 .666 .684 .725 1.047
.857 .915 .994 .736 .978
.666 .621 .803 .928
.365 .811 .637
.805 .736
1.092
13
WAKEFIELD, BRADLEY, DOUGHTIE, AND KRAFT
856
TABLE S 176 DISTANCE COMPARISONS FOR NONOVERLAPPING SCALES Long distances Middle distances Si-Hs Si-D Si-IIy Si-Pa. Si-Pt Si-Sc Sc-Hs Sc-D Sc-Hy Pt-Hs Pt-D Pt-Hy Par-Hs Pa-D Pa-Hy Ma-IIs Ma-D Ma-Hy Sc-Pd Pi-Pi Pa-Pd Ma-Pd
Short distances
Si-Ma
Si-Pd
Ily-D
D-Hs
Hy-Hs
Sc-Pt
Pt-Pa
Sc-Pa
1
0 1 0 0 1 1 1 1 0
1 0 1 0 0 0 0
1 1 1 1 1 1 0 0
0 0
1 0
1 0 0 0 0 0
0 0 0 0
1 1 1 1 1 1 1' 1
1 0 1 0 0 0 0 0
0 0
0
0
0 0
1 1
0 0
0
1
0
1
0
0 0 0 0
0
1 1 1 1
0 0 0 0
0 0
0
0 0 0 0 0 0 0 0 0
0 1
0 0
1
0
1
0 1 1 1 1 1 0 1 1 0
1 1
1 1 1 1 1 1
0
1
1 1
0
1
1
0
1 1
0
1
0 0 0
0
1
1 1 1
1
1
1 1
0
1
1
1
1 1
0
1 1
1
1
0
1 1
0
0
1
1 1
0
0
1 1 1
1
1
1 1
1
1
1
1
• .68, ns.
Note. 1 = correct; 0 = incorrect; number correct
TABLE 6 176 DISTANCE COMPARISONS TOR OVERLAPPING SCALES Long distances Middle distances
Si-Ma
Si-Pd
Si-Hs Si-D Si-Hy Si-Pa Si-Pt Si-Sc Sc-Hs Sc-D Sc-Hy Pt-Hs Pt-D Pt-Hy Pa-Hs Pa-D Pa-Hy Ma-Hs Ma-D Ma-Hy Sc-Pd Pt-Pd Pa-Pd Ma-Pd
1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1
0
1 1 1 1 1
Short distances
Hy-D
D-Hs
1 0 1 1 0 0 0 0 1 0 0 1 0 0 0 1 1 1 0 0 0 0
1 0 1 1
0
1 1
0
1
0 0
1 1
0
1 1 1 1
0
0 0
1
Note. 1 = correct; 0 = incorrect; number correct = 141; z - 7.92, p < .001.
Hy-Hs
Sc-Pt
Pt-Pa,
Sc-Pa
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 I 1
1 0 1 1 0 1 1 1 1 1 0
1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0
1 1
0
1 1 1 1 1
0 0
1
1
OVERLAPPING AND NONOVERLAPPING ITEMS ON THE MMPI latent variable. Thus, theoretical interrelationships among latent variables should be observable only in items occurring on more than one scale. No theoretical interrelationship should be observable in the items occurring on only one scale. The results of the present study conformed to the two previous statements. They showed an extremely strong pattern of theoretical interrelationships for the overlapping items and no significant pattern of interrelationships for thenonoverlapping items. An early criticism of the MMPI (Benton, 1949) was that it was inadequate for differentiating between different categories of abnormality even though it did discriminate between normals and abnormals and indicate severity of abnormality. The present results suggest that the mixture in each scale of. (a) overlapping items measuring global personality dimensions and (b) nonoverlapping items possibly measuring finer qualities of particular categories of abnormality may impede the differential diagnosis of personality with the MMPI. The distinction between the two types of items contained in the MMPI scales and the different theoretical levels of personality each type measures leads to a possible method of improving the diagnostic value of the MMPI. A hierarchical approach to personality measurement consistent with Eysenck's (1947) theory of personality is suggested. The first stage in this approach would rely on scales scored only for overlapping items to determine the global personality dimension -(neuroticism or psychoticism) that best characterized an individual. The second stage would rely on scales scored for nonoverlapping items to make finer distinctions within neuroticism or within psychoticism. Although the present results are consistent with this two-stage process of personality diagnosis, a great deal of research with carefully chosen criterion groups is necessary to add empirical substance to this theoretically derived technique for improving differential personality diagnosis.
857
REFERENCES Adams, D. K., & Horn, J, L. Nonoverlapping keys for the MMPI scales. Journal of Consulting Psychology, 1965, 29, 284. Anderson, H. E., & Bashaw, W. L. Further comments on. the internal structure of the MMPI. Psychological Bulletin, 1966, 66, 211-213. Benton, A. L. Review of the MMPI. In 0. K. Euros (Ed.), Third mental measurements yearbook. New Brunswick, N J.: Rutgers University Press, 1949. Dahlstrom, W. G., Welsh, G. S., & Dahlstrom, L. E. MMPI handbook, Clinical interpretation (Vol. 1) (Rev. ed.). Minneapolis: University of Minnesota Press, 1972. Eysenck, H. J. Dimensions of personality. London: Kegan, Paul, 1947. Eysenck, H. J. Review of the MMPI. In O. K. Euros (Ed.), Third mental measurements yearbook. New Brunswick, N.J.: Rutgers University Press, 1949. Eysenck, H. J. A dimensional system of psychodiagnostics. In A. R. Mahrer (Ed.), New approaches to personality classification. New York: Columbia University Press, 1970. Guilford, J. P. When not to factor analyze. Psychological Bulletin, 19S2, 49, 26-37. Harman, H. H. Modern factor analysis. Chicago: University of Chicago Press, 1970. Shure, G. H., & Rogers, M. S. Note of caution on the factor analysis of the MMPI. Psychological Bulletin, 1965, 63, 14-18. Siegel, S. Nonparametric statistics. New York: McGrawHill, 1956. Stein, K. B. The TSC scales: The outcome of a cluster analysis of the 550 MMPI items. In P. McReynolds (Ed.), Advances in psychological assessment (Vol. 1). Palo Alto, Calif.: Science and Behavior Books, 1968. Wakefield, J. A., Jr., & Doughtie, E, B. The geometric relationship between Holland's personality typology and the Vocational Preference Inventory. Journal of Counseling Psychology, 1973, 20, 513-518. Wakefield, J. A., Jr., Yom, B. L., Bradley, P. E., Doughtie, E. B., Cox, J. A., & Kraft, I. A. Eysenck's personality dimensions: Model for the MMPI. British Journal of Social and Clinical Psychology, 1974, 13, 413-420. Weiss, D. J. Factor analysis and counseling research. Journal of Counseling Psychology, 1970, 17, 477-485. Welsh, G. S. Factor dimensions A and R. In G. S. Welsh & W. G. Dahlstrom (Eds.), Basic readings on the MMPI in psychology and medicine. Minneapolis: University of Minnesota Press, 1956. Wheeler, W. M., Little, K. B., & Lehner, G. F. J. The internal structure of the MMPI. Journal of Consulting Psychology, 1951,15, 134-141. (Received March 31,1975)
