Psychological Reports, 1992, 71, 1264-1266.
O Psychological Reports 1992
CONSTRUCT VALIDITY O F T H E COMMITMENT T O CAREER CHOICES SCALE AMONG LATE ADOLESCENTS: ANOTHER LOOK MICHAEL V. ELLIS AND DAVID L. BLUSTEIN Department of Corrnseling Psychology University a t Alba y State University of New York Summary.-In response to Stead and Watson (1992), we reassessed the construct validity of the Commitment ro Career Choices Scale using an age-constrained sample from our aggregate data base. A confirmatory factor analysis yielded findings comparable to Stead and Watson's results, thereby suggesting that their findings were likely due to using a homogeneous sample with truncated scores. Given the theoretical importance of measuring the full range of the two commitment to career choices concinua, it seems premature to delete items from the scales for research with South African university students. Alternatively, we encourage the use of norms to interpret scores from the scales.
The work by Stead and Watson (1992) offers an important contribution towards testing the validity of inferences from the Commitment to Career Choices Scale (Blustein, Ellis, & Devenis, 1989). Although they concluded that their data "support the construct validity of the scale" (p. 1005), for South African research, they recommend omitting Items 21, 15, and 16 from the Vocational Exploration and Commitment and the Tendency to Foreclose scales which comprise the Commitment to Career Choices Scale. Because these scales were developed adhering to the unificationist view of vhdity, which emphasizes the preeminence of theory and traditional methods of hypothesis testing in test validation (Ellis & Blustein, 1991a, 1991b; Messick, 1989), this recommendation is potentially misleading and inconsistent with the commitment to career choices process model for which the scales were developed to measure and test. We believe that Stead and Watson's study was influenced by theoretical, methodological, and statistical factors. Theoretically, Stead and Watson assumed that the commitment to career choices process somehow differs for Caucasian South Africans when compared to Americans. Unfortunately, they did not explicate and test these differences. Methodologically, the sample of 19-year-old, first-year university students (comparable to first- and secondyear U.S. university students) taps a constricted range of the vocational exploration and commitment continuum. That is, the homogeneous sample was most likely restricted to the exploration phase of the commitment to career 'Address correspondence to M. V. Ellis, Department of Counselin Psychology, University at Albany, State University of New York, 1400 Washington Avenue, ~ l % a n yN, Y 12222.
1265
THEORY-DRIVEN CONSTRUCT VALIDITY
choices process, thereby truncating scores on the Vocational Exploration and Commitment Scale (i.e., items tapping the commitment phase would not be expected to have high factor loadings for this sample). Statistically, maximum likelihood confirmatory factor analysis is considered more appropriate for model (theory) testing than the exploratory principal factor analysis used by Stead and Watson (see references in Blustein, et af., 1989). These two procedures typically yield disparate factor solutions; indeed, their homogeneous sample may have exacerbated the differences in the results obtained by the two procedures. Although any one of these factors could readily explain their findings, we hypothesized that Stead and Watson's results were largely due to the methodological factor (i.e., an equally constrained U.S. sample would yield results similar to Stead and Watson's). To test this hypothesis, we homogenized our aggregate data base yielding a sample comprised of 493 Caucasian freshmen and sophomores who had a median age of 19.0 (M = 19.8, SD = 4.0) and mean grade point average of 2.91 (SD = .53) of 4.0. The sample included 59.3% women and 38.9% freshmen. A confirmatory factor analysis configured as a two-factor congeneric (oblique) model yielded a X,,,Z = 1219.60 ( p < .001, p = .73, GFI = .83). The factor correlation was -.41, with the two factors accounting for 36.5% of the variance. Cronbach's alphas for the Tendency to Foreclose Scale and the Vocational Exploration and Commitment Scale were .80 and .91, respectively. One Tendency to Foreclose Scale item (No. 16) and three Vocational Exploration and Commitment Scale items (Nos. 3, 7, and 21) had factor loadings less than .40 (.28, .38, 2 6 , and .36, respectively). After modification procedures, two Vocational Exploration and Commitment Scale items (Nos. 21 and 24) and one Tendency to Foreclose Scale item (No. 16) loaded greater than .25 on the opposing factor (.26, -.27, and .27, respectively). Our results are comparable to Stead and Watson's (1992) in that similar items did not meet the specified criteria. It seems reasonable to conclude that the use of an age-constricted Caucasian sample served to alter the capacity of the Commitment to Career Choices Scales to tap fully the range of relevant attitudes and behaviors. Therefore, given the theoretical importance of the items in measuring the two continua of commitment to career choices, it seems premature to delete these items from either scale. AIternatively, we encourage the development and use of norms to aid in interpreting scores from the scales (cf. Blustein, et al., 1989, p. 361). Consistent with the developmental-contextual paradigm in career development theory (e.g., Vondracek, Lerner, & Schulenberg, 1986), we believe that progress in research may be optimally obtained if investigators examine the theoretical meaning of their results in light of the relevant contextual aspects of their study. We also encourage investigators to conduct theory-
*
1266
M. V. ELLIS & D. L. BLUSTEIN
driven validation studies that use rigorous statistical procedures such as confirmatory factor analysis rather than atheoretical, exploratory research (see references in Blustein, et al., 1989). REFERENCES BLUSTEIN,D. L., ELLIS,M. V., & DEVENIS,L. E. (1989) The development and validation of a two-dimensional model of the commitment to career choices process. Journal of Vocational Behavior, 35, 342-378. (Monograph) ELLIS,M. V., & BLUSTEIN,D. L. (1991a) Developing and using educational and psychological tests and measures: the unificationist perspective. Journal of Counseling and Development, 69, 550-555. ELLIS, M. V., & BLUSTEIN,D. L. (1991b) The unificationist view: a context for validity. Journal of Counseling and Development, 69, 561-563. MESSICK,S. (1989) Validity. In R. L. Linn (Ed.), Educational measurement. (3rd ed.) New York: Macmillan. Pp. 13-103. STFAD, G. B., & WATSON,M. B. (1992) Construct validity of the Commitment co Career Choices Scale among South African university students. Psychological Repor&, 70, 10051006. VONDRACEK, F. W., LERNER,R. M., & SCHULENBERG, J. E. (1986) Career development: a lijespan developmental approach. Hillsdale, N J : Erlbaum.
Accepted Nouember 9, 1992.
O Psychological Reports 1992
CONSTRUCT VALIDITY O F T H E COMMITMENT T O CAREER CHOICES SCALE AMONG LATE ADOLESCENTS: ANOTHER LOOK MICHAEL V. ELLIS AND DAVID L. BLUSTEIN Department of Corrnseling Psychology University a t Alba y State University of New York Summary.-In response to Stead and Watson (1992), we reassessed the construct validity of the Commitment ro Career Choices Scale using an age-constrained sample from our aggregate data base. A confirmatory factor analysis yielded findings comparable to Stead and Watson's results, thereby suggesting that their findings were likely due to using a homogeneous sample with truncated scores. Given the theoretical importance of measuring the full range of the two commitment to career choices concinua, it seems premature to delete items from the scales for research with South African university students. Alternatively, we encourage the use of norms to interpret scores from the scales.
The work by Stead and Watson (1992) offers an important contribution towards testing the validity of inferences from the Commitment to Career Choices Scale (Blustein, Ellis, & Devenis, 1989). Although they concluded that their data "support the construct validity of the scale" (p. 1005), for South African research, they recommend omitting Items 21, 15, and 16 from the Vocational Exploration and Commitment and the Tendency to Foreclose scales which comprise the Commitment to Career Choices Scale. Because these scales were developed adhering to the unificationist view of vhdity, which emphasizes the preeminence of theory and traditional methods of hypothesis testing in test validation (Ellis & Blustein, 1991a, 1991b; Messick, 1989), this recommendation is potentially misleading and inconsistent with the commitment to career choices process model for which the scales were developed to measure and test. We believe that Stead and Watson's study was influenced by theoretical, methodological, and statistical factors. Theoretically, Stead and Watson assumed that the commitment to career choices process somehow differs for Caucasian South Africans when compared to Americans. Unfortunately, they did not explicate and test these differences. Methodologically, the sample of 19-year-old, first-year university students (comparable to first- and secondyear U.S. university students) taps a constricted range of the vocational exploration and commitment continuum. That is, the homogeneous sample was most likely restricted to the exploration phase of the commitment to career 'Address correspondence to M. V. Ellis, Department of Counselin Psychology, University at Albany, State University of New York, 1400 Washington Avenue, ~ l % a n yN, Y 12222.
1265
THEORY-DRIVEN CONSTRUCT VALIDITY
choices process, thereby truncating scores on the Vocational Exploration and Commitment Scale (i.e., items tapping the commitment phase would not be expected to have high factor loadings for this sample). Statistically, maximum likelihood confirmatory factor analysis is considered more appropriate for model (theory) testing than the exploratory principal factor analysis used by Stead and Watson (see references in Blustein, et af., 1989). These two procedures typically yield disparate factor solutions; indeed, their homogeneous sample may have exacerbated the differences in the results obtained by the two procedures. Although any one of these factors could readily explain their findings, we hypothesized that Stead and Watson's results were largely due to the methodological factor (i.e., an equally constrained U.S. sample would yield results similar to Stead and Watson's). To test this hypothesis, we homogenized our aggregate data base yielding a sample comprised of 493 Caucasian freshmen and sophomores who had a median age of 19.0 (M = 19.8, SD = 4.0) and mean grade point average of 2.91 (SD = .53) of 4.0. The sample included 59.3% women and 38.9% freshmen. A confirmatory factor analysis configured as a two-factor congeneric (oblique) model yielded a X,,,Z = 1219.60 ( p < .001, p = .73, GFI = .83). The factor correlation was -.41, with the two factors accounting for 36.5% of the variance. Cronbach's alphas for the Tendency to Foreclose Scale and the Vocational Exploration and Commitment Scale were .80 and .91, respectively. One Tendency to Foreclose Scale item (No. 16) and three Vocational Exploration and Commitment Scale items (Nos. 3, 7, and 21) had factor loadings less than .40 (.28, .38, 2 6 , and .36, respectively). After modification procedures, two Vocational Exploration and Commitment Scale items (Nos. 21 and 24) and one Tendency to Foreclose Scale item (No. 16) loaded greater than .25 on the opposing factor (.26, -.27, and .27, respectively). Our results are comparable to Stead and Watson's (1992) in that similar items did not meet the specified criteria. It seems reasonable to conclude that the use of an age-constricted Caucasian sample served to alter the capacity of the Commitment to Career Choices Scales to tap fully the range of relevant attitudes and behaviors. Therefore, given the theoretical importance of the items in measuring the two continua of commitment to career choices, it seems premature to delete these items from either scale. AIternatively, we encourage the development and use of norms to aid in interpreting scores from the scales (cf. Blustein, et al., 1989, p. 361). Consistent with the developmental-contextual paradigm in career development theory (e.g., Vondracek, Lerner, & Schulenberg, 1986), we believe that progress in research may be optimally obtained if investigators examine the theoretical meaning of their results in light of the relevant contextual aspects of their study. We also encourage investigators to conduct theory-
*
1266
M. V. ELLIS & D. L. BLUSTEIN
driven validation studies that use rigorous statistical procedures such as confirmatory factor analysis rather than atheoretical, exploratory research (see references in Blustein, et al., 1989). REFERENCES BLUSTEIN,D. L., ELLIS,M. V., & DEVENIS,L. E. (1989) The development and validation of a two-dimensional model of the commitment to career choices process. Journal of Vocational Behavior, 35, 342-378. (Monograph) ELLIS,M. V., & BLUSTEIN,D. L. (1991a) Developing and using educational and psychological tests and measures: the unificationist perspective. Journal of Counseling and Development, 69, 550-555. ELLIS, M. V., & BLUSTEIN,D. L. (1991b) The unificationist view: a context for validity. Journal of Counseling and Development, 69, 561-563. MESSICK,S. (1989) Validity. In R. L. Linn (Ed.), Educational measurement. (3rd ed.) New York: Macmillan. Pp. 13-103. STFAD, G. B., & WATSON,M. B. (1992) Construct validity of the Commitment co Career Choices Scale among South African university students. Psychological Repor&, 70, 10051006. VONDRACEK, F. W., LERNER,R. M., & SCHULENBERG, J. E. (1986) Career development: a lijespan developmental approach. Hillsdale, N J : Erlbaum.
Accepted Nouember 9, 1992.
