Perceptual and MotorSkillr, 1990, 71, 827-831.
O Perceptual and Motor Skills 1990
RAW SCORES VS PERCENTAGE CONVERSIONS I N FACTORIAL SOLUTIONS FOR PROJECTIVE T E S T VARIABLES " ' EDWIN E. WAGNER
MARK J. HILSENROTH
Forest Institute, Huntsville, Ahbama
University of Tennessee
HARRY J. SIVEC Ohio University Summary.-Factor analytic solutions for raw scores and corresponding percentage conversion scores derived from Rorschach and Hand Test variables were direcrly com~ a r e d .Number of factors, total and individual communalities, and test overlap were sufficiently different to question either approach as preferable for summarizing interrelationships among projective tests and test variables. Most of the substantial loadings tended to be either artifactual, relatively uninformative, or appeared on one factor analysis but not the other.
It has been shown (Wagner, Prospero, & Alexander, 1984) that the use of percentage scores instead of raw scores produces different discriminant analysis solutions and, perforce, divergent interpretations when dealing with data obtained from those projective techniques on which the total number of responses (R) varies from subject to subject. The present study was designed to assess whether the same kind of inconsistency appears in factor analyses, rendering either a raw score or a percentage score solution open to question. To be sure, other investigators have commented on artifactual influences in factoring projective data (e.g., Coan, 1956; Stotsky, 1957), but a review of the literature indicates no direct and exhaustive comparison of how raw score and percentage score usage differentially affects factor analytic solutions for identical variables. While there are many other thorny issues associated with factoring scores from Rorschach-type tests, our research focuses specifically on this problem. The intent of this exercise, therefore, was to undertake a full blown demonstration of the irreconcilable anomalies which emerge when comparing factorial solutions derived from raw scores and percentages. We are not trying to justify either procedure.
METHOD To assemble a reasonably representative clinical outpatient sample, previously diagnosed subjects obtained from the files of one of the authors were selected on the basis of belonging to three large, major diagnostic classifications: organic brain syndromes ( 0 ) , schizophrenics (S), and personality 'Appreciation is extended to Dr. Ralph A. Alexander for his help with the statistical analyses. 'Address correspondence to E. E. Wagner, Dean, Forest Institute, 261 1 Leeman Ferry Road, Huntsville, AL 35801-5611.
828
E. E. WAGNER, E T A L .
disorders (P). For purposes of this demonstration equal numbers of subjects (n = 76) were included in each of the three groups. For the entire 228 cases the mean age was 28.9 yr., with a SD of 10.2 yr. One hundred sixteen were single, 74 married, 35 divorced, and 3 widowed. Average years of education was 11.3 (SD2.2). Mean WAIS Full Scale IQ was 93.0 (SD 15.0). Some major Rorschach and Hand Test variables and variable combinations were factored together to demonstrate how the relationships between projective tests as well as the number of factors extracted might be affected by whether raw scores or percentages are used. For both sets of data the factorial procedure was discontinued when skree plots (Cattell, 1966) demonstrated significant breaks. Orthogonal (varimax) rotation was applied to both sets of data. To lessen the pervasive influence of the productivity factor, R was not included in the raw score factoring.
Fortuitously, the skree breaking points also coincided with Kaiser's (1960) rule which only retains factors when the eigenvalue exceeds 1.0. Since
there was agreement across two criteria, it seems reasonably certain that the two sets of factors extracted are not artifacts of different or arbitrary decision rules regarding the cessation of factor extractions. However, despite the fact that identical factoring processes were utilized, five significant factors emerged for the raw scores compared to seven for the percentage scores; see Table 1. TABLE
1
VARIABLEFACTOR LOADINGS FORRAW SCORESA N D PERCENTAGE CONVERSIONS (N = 228)
Variable
Raw Scores: Factors
I
I1
I11
IV
v
I
Percentage Conversion: Factors I1 I11 IV v VI VII
Rorschach W + WS
D DS+S All other area scores (Di,De, Dr, Dx, etc.)
FAIL Deviant Verbalizations
F F*, F A+M H+Hd obj anat
(continued on next page)
FACTOR SOLUTIONS: RAW SCORES VS PERCENTAGE CONVERSIONS
829
TABLE 1 (CONT'D) VARIABLEFACTOR LOADINGS FORRAWSCORES AND PERCENTAGE CONVERSIOAS (N = 228)
Variable
Raw
I ernbl + app
blood + stain +imp+exp P
AIRT M FM
EC Cc
I1
Scores: Factors I11 IV
v
525
453 -001 -080 -156
084 244 090 394 130 185 356
505 040 102 635 371 748 657
104 -037 083 -096 -079 -075 192 147 187 134 -087 -112 010 -061 177 023 025
017 033 012 407 289 025 216 128 039 096 234 321 -074 -305 103 026 003
I
Percentage Conversion: Factors I1 I11 IV v VI VII
-088 155 172 578 -133 039 -051 760 -094 088 -067 -420 359 208 025 082 142 252 129 155 056
Hand Test
AFF DEP COM EXH DIR AGG
ACQ ACT PAS TEN
CRIP FEAR DES FAIL BIZ
AIRT H-L
240 166 394 021 391 486 133 375 -002 197 182 180 -605 -467 -130 -030 -017
-007 119 -148 -139 083 -069 019 055 014 -023 -001 084 -106 -037 061 761 839
-287 263 087 -137 038 -097 -054 057 245 273 -030 -003 349 -180 086 039 -099
Note.-Decimals omitted. Total communality was 15.55 for the raw scores and 17.20 for the percentage scores, with communalities sometimes quite similar but at other times varying considerably for individual variables. For example, the total communality for the Hand Test Activity (ACT) variable was only ,185 for the raw scores but .699 for the percentage scores. I n other cases, however, communalities were comparable, e.g., .770 and .756 for the Rorschach form (F) variable. I n general,-commun&ties were higher for the Rorschach variables than for the Hand Test variables which may indicate more redundancy, not necessarily undesirable in a diagnostic technique. As expected, the artifactual productivity loadings for the raw scores plus reciprocal loadings due to percentage conversions were clearly evidenced. For example, for the raw scores Rorschach D and F were highly loaded on Factor
830
E. E. WAGNER, ET AL. TABLE 2 INDIVDUALRORSCHACH AND HANDTESTVARIABLE COMMLJNAUTIES FOR RAW SCORESAND PERCENTAGE CONVERSIONS (N = 228) Variable
Raw Score
% Conversion
Rorschach
W + WS D DS + S All other area scores (Di,De,Dr,Dx, etc.) FAIL Deviant Verbalizations
F F*,FA+& H+Hd obi anat f l + p l + N + bot + water + food embl + app blood + stain + imp t exp P
AIRT M FM
EC Ec Hand Test
AFF DEP COM EXH DIR AGG ACQ ACT PAS TEN CRIP FEAR DES FAIL BIZ AIRT H-L
I (.79 and .77, respectively) as previously reported by other researchers (Wittenborn, 1950; Cox, 1951; Schori & Thomas, 1972). Likewise, for the percentage solutions, Wand D were highly but inversely correlated on Factor
FACTOR SOLUTIONS: RAW SCORES VS PERCENTAGE CONVERSIONS
831
I11 (W + WS = .85, D = -.62) as described by Adcock (1951), Wishner (1959), and Geertsma (1962). The interrelationships between Rorschach and Hand Test variables for the two sets of factors were difficult to interpret and inimical to the establishment of consistent overlaps and construct validity for the two instruments. For example, the Rorschach Popular (P) and Hand Test Description (DES) variables loaded substantially and inversely on raw score Factor I11 which appears to be reflecting some kind of intactness/pathology vector, but unfortunately no such confirming association was noted on any of the percentage conversion factors. O n the other hand, reaction times for both the Rorschach and Hand Test were the major components of and obviously explained raw score Factor IV and percentage score Factor 11. I t seems doubtful that innovative rotations would bring about better agreement between the factor sets since much of the variance is obviously accounted for by either artifactual effects or substantial but relatively uninformative relationships such as reaction times. Furthermore, we also computed normalized McCall T-scores. Predictably, the factorial structure and individual and total communalities for the normalized variables were virtually identical with that of the percentage conversion variables. It can be concluded that, based on differences in the raw score vs percentage discrepancies alone, factor analysis of Rorschach-type projective techniques should be viewed with caution, even skepticism. I t also seems prudent to factor analyze both the raw scores and percentage scores before ascribing undue significance to factor loadings emerging from either method. REFERENCES ADCOCK,C. J. (1951) A factorial approach to Rorschach interpretations. Journal of General Psychology, 44, 261-272. C A ~ R. , B. (1966) The skree test for the number of Factors. Mzlltivariate Behavioral Research, 1, 245-276. COAN, R. (1956) A factor analysis of Rorschach determinants. Journal of Projective Techniques, 20, 280-287.
Cox, S. M. (1951) A factorial study of the Rorschach responses of normal and maladiusted boys. Journal of Genetic Psychology, 79, 95-115. GEERTSMA, R. H. (1962) Factor analysis of Rorschach scoring categories for a population of normal subjects. Journal of Consulting Psychology, 26, 20-25. KAISER, H. F. (1960) The application of electronic computers to factor analysis. Educational and Psychological Measrrrement, 20, 141-151. SCHORI,T. R., & THOMAS,C. D. (1972) The Rorschach test: an image analysis. Journal of Clinical Psychology, 27, 195-199. STOTSKY,B. A. (1957) Factor analysis of Rorschach scores of schizophrenics. Journal of Clinical Psychology, 13, 275-278. WAGNER, E. E., PROSPERO, M. K., & ALEXANDEX, R. A. (1984) Percenc3ge conversions as a method of enhancing clinical discriminations. Journal of Personal~ty Assessment, 48, 617-619.
WISHNER, J. R. (1959) Factor analysis of Rorschach scoring categories and first response times in normals. Journal of Consulting Psychology, 23, 406-413. WITTENBORN, J. R. (1950) A factor analysis of Rorschach scoring categories. Journal of Consulting Psychology, 14, 261-267. Accepted October 8, 1990.
O Perceptual and Motor Skills 1990
RAW SCORES VS PERCENTAGE CONVERSIONS I N FACTORIAL SOLUTIONS FOR PROJECTIVE T E S T VARIABLES " ' EDWIN E. WAGNER
MARK J. HILSENROTH
Forest Institute, Huntsville, Ahbama
University of Tennessee
HARRY J. SIVEC Ohio University Summary.-Factor analytic solutions for raw scores and corresponding percentage conversion scores derived from Rorschach and Hand Test variables were direcrly com~ a r e d .Number of factors, total and individual communalities, and test overlap were sufficiently different to question either approach as preferable for summarizing interrelationships among projective tests and test variables. Most of the substantial loadings tended to be either artifactual, relatively uninformative, or appeared on one factor analysis but not the other.
It has been shown (Wagner, Prospero, & Alexander, 1984) that the use of percentage scores instead of raw scores produces different discriminant analysis solutions and, perforce, divergent interpretations when dealing with data obtained from those projective techniques on which the total number of responses (R) varies from subject to subject. The present study was designed to assess whether the same kind of inconsistency appears in factor analyses, rendering either a raw score or a percentage score solution open to question. To be sure, other investigators have commented on artifactual influences in factoring projective data (e.g., Coan, 1956; Stotsky, 1957), but a review of the literature indicates no direct and exhaustive comparison of how raw score and percentage score usage differentially affects factor analytic solutions for identical variables. While there are many other thorny issues associated with factoring scores from Rorschach-type tests, our research focuses specifically on this problem. The intent of this exercise, therefore, was to undertake a full blown demonstration of the irreconcilable anomalies which emerge when comparing factorial solutions derived from raw scores and percentages. We are not trying to justify either procedure.
METHOD To assemble a reasonably representative clinical outpatient sample, previously diagnosed subjects obtained from the files of one of the authors were selected on the basis of belonging to three large, major diagnostic classifications: organic brain syndromes ( 0 ) , schizophrenics (S), and personality 'Appreciation is extended to Dr. Ralph A. Alexander for his help with the statistical analyses. 'Address correspondence to E. E. Wagner, Dean, Forest Institute, 261 1 Leeman Ferry Road, Huntsville, AL 35801-5611.
828
E. E. WAGNER, E T A L .
disorders (P). For purposes of this demonstration equal numbers of subjects (n = 76) were included in each of the three groups. For the entire 228 cases the mean age was 28.9 yr., with a SD of 10.2 yr. One hundred sixteen were single, 74 married, 35 divorced, and 3 widowed. Average years of education was 11.3 (SD2.2). Mean WAIS Full Scale IQ was 93.0 (SD 15.0). Some major Rorschach and Hand Test variables and variable combinations were factored together to demonstrate how the relationships between projective tests as well as the number of factors extracted might be affected by whether raw scores or percentages are used. For both sets of data the factorial procedure was discontinued when skree plots (Cattell, 1966) demonstrated significant breaks. Orthogonal (varimax) rotation was applied to both sets of data. To lessen the pervasive influence of the productivity factor, R was not included in the raw score factoring.
Fortuitously, the skree breaking points also coincided with Kaiser's (1960) rule which only retains factors when the eigenvalue exceeds 1.0. Since
there was agreement across two criteria, it seems reasonably certain that the two sets of factors extracted are not artifacts of different or arbitrary decision rules regarding the cessation of factor extractions. However, despite the fact that identical factoring processes were utilized, five significant factors emerged for the raw scores compared to seven for the percentage scores; see Table 1. TABLE
1
VARIABLEFACTOR LOADINGS FORRAW SCORESA N D PERCENTAGE CONVERSIONS (N = 228)
Variable
Raw Scores: Factors
I
I1
I11
IV
v
I
Percentage Conversion: Factors I1 I11 IV v VI VII
Rorschach W + WS
D DS+S All other area scores (Di,De, Dr, Dx, etc.)
FAIL Deviant Verbalizations
F F*, F A+M H+Hd obj anat
(continued on next page)
FACTOR SOLUTIONS: RAW SCORES VS PERCENTAGE CONVERSIONS
829
TABLE 1 (CONT'D) VARIABLEFACTOR LOADINGS FORRAWSCORES AND PERCENTAGE CONVERSIOAS (N = 228)
Variable
Raw
I ernbl + app
blood + stain +imp+exp P
AIRT M FM
EC Cc
I1
Scores: Factors I11 IV
v
525
453 -001 -080 -156
084 244 090 394 130 185 356
505 040 102 635 371 748 657
104 -037 083 -096 -079 -075 192 147 187 134 -087 -112 010 -061 177 023 025
017 033 012 407 289 025 216 128 039 096 234 321 -074 -305 103 026 003
I
Percentage Conversion: Factors I1 I11 IV v VI VII
-088 155 172 578 -133 039 -051 760 -094 088 -067 -420 359 208 025 082 142 252 129 155 056
Hand Test
AFF DEP COM EXH DIR AGG
ACQ ACT PAS TEN
CRIP FEAR DES FAIL BIZ
AIRT H-L
240 166 394 021 391 486 133 375 -002 197 182 180 -605 -467 -130 -030 -017
-007 119 -148 -139 083 -069 019 055 014 -023 -001 084 -106 -037 061 761 839
-287 263 087 -137 038 -097 -054 057 245 273 -030 -003 349 -180 086 039 -099
Note.-Decimals omitted. Total communality was 15.55 for the raw scores and 17.20 for the percentage scores, with communalities sometimes quite similar but at other times varying considerably for individual variables. For example, the total communality for the Hand Test Activity (ACT) variable was only ,185 for the raw scores but .699 for the percentage scores. I n other cases, however, communalities were comparable, e.g., .770 and .756 for the Rorschach form (F) variable. I n general,-commun&ties were higher for the Rorschach variables than for the Hand Test variables which may indicate more redundancy, not necessarily undesirable in a diagnostic technique. As expected, the artifactual productivity loadings for the raw scores plus reciprocal loadings due to percentage conversions were clearly evidenced. For example, for the raw scores Rorschach D and F were highly loaded on Factor
830
E. E. WAGNER, ET AL. TABLE 2 INDIVDUALRORSCHACH AND HANDTESTVARIABLE COMMLJNAUTIES FOR RAW SCORESAND PERCENTAGE CONVERSIONS (N = 228) Variable
Raw Score
% Conversion
Rorschach
W + WS D DS + S All other area scores (Di,De,Dr,Dx, etc.) FAIL Deviant Verbalizations
F F*,FA+& H+Hd obi anat f l + p l + N + bot + water + food embl + app blood + stain + imp t exp P
AIRT M FM
EC Ec Hand Test
AFF DEP COM EXH DIR AGG ACQ ACT PAS TEN CRIP FEAR DES FAIL BIZ AIRT H-L
I (.79 and .77, respectively) as previously reported by other researchers (Wittenborn, 1950; Cox, 1951; Schori & Thomas, 1972). Likewise, for the percentage solutions, Wand D were highly but inversely correlated on Factor
FACTOR SOLUTIONS: RAW SCORES VS PERCENTAGE CONVERSIONS
831
I11 (W + WS = .85, D = -.62) as described by Adcock (1951), Wishner (1959), and Geertsma (1962). The interrelationships between Rorschach and Hand Test variables for the two sets of factors were difficult to interpret and inimical to the establishment of consistent overlaps and construct validity for the two instruments. For example, the Rorschach Popular (P) and Hand Test Description (DES) variables loaded substantially and inversely on raw score Factor I11 which appears to be reflecting some kind of intactness/pathology vector, but unfortunately no such confirming association was noted on any of the percentage conversion factors. O n the other hand, reaction times for both the Rorschach and Hand Test were the major components of and obviously explained raw score Factor IV and percentage score Factor 11. I t seems doubtful that innovative rotations would bring about better agreement between the factor sets since much of the variance is obviously accounted for by either artifactual effects or substantial but relatively uninformative relationships such as reaction times. Furthermore, we also computed normalized McCall T-scores. Predictably, the factorial structure and individual and total communalities for the normalized variables were virtually identical with that of the percentage conversion variables. It can be concluded that, based on differences in the raw score vs percentage discrepancies alone, factor analysis of Rorschach-type projective techniques should be viewed with caution, even skepticism. I t also seems prudent to factor analyze both the raw scores and percentage scores before ascribing undue significance to factor loadings emerging from either method. REFERENCES ADCOCK,C. J. (1951) A factorial approach to Rorschach interpretations. Journal of General Psychology, 44, 261-272. C A ~ R. , B. (1966) The skree test for the number of Factors. Mzlltivariate Behavioral Research, 1, 245-276. COAN, R. (1956) A factor analysis of Rorschach determinants. Journal of Projective Techniques, 20, 280-287.
Cox, S. M. (1951) A factorial study of the Rorschach responses of normal and maladiusted boys. Journal of Genetic Psychology, 79, 95-115. GEERTSMA, R. H. (1962) Factor analysis of Rorschach scoring categories for a population of normal subjects. Journal of Consulting Psychology, 26, 20-25. KAISER, H. F. (1960) The application of electronic computers to factor analysis. Educational and Psychological Measrrrement, 20, 141-151. SCHORI,T. R., & THOMAS,C. D. (1972) The Rorschach test: an image analysis. Journal of Clinical Psychology, 27, 195-199. STOTSKY,B. A. (1957) Factor analysis of Rorschach scores of schizophrenics. Journal of Clinical Psychology, 13, 275-278. WAGNER, E. E., PROSPERO, M. K., & ALEXANDEX, R. A. (1984) Percenc3ge conversions as a method of enhancing clinical discriminations. Journal of Personal~ty Assessment, 48, 617-619.
WISHNER, J. R. (1959) Factor analysis of Rorschach scoring categories and first response times in normals. Journal of Consulting Psychology, 23, 406-413. WITTENBORN, J. R. (1950) A factor analysis of Rorschach scoring categories. Journal of Consulting Psychology, 14, 261-267. Accepted October 8, 1990.
