Psychological Reports: Measures & Statistics 2013, 113, 2, 486-489. © Psychological Reports 2013
GENDER INVARIANCE OF THE COLLEGE STUDENT STRESS SCALE1 RONALD C. FELDT AND CHRISTINA UPDEGRAFF Mount Mercy University Summary.—Assessment of perceived stress may be an important prerequisite to deployment of effective coping in efforts to help college students adjust to academic and social demands of college. The study examined the extent to which a seven-item measure of the College Student Stress Scale is invariant across gender. Results indicated invariance of factor loadings, factor variance, and all but one item intercept. No statistically significant gender difference was observed between latent variable means.
The inability to manage the stress response in college is associated with poor college adjustment, depression, and reduced life satisfaction (Segrin, Hanzal, Donnerstein, Taylor, & Domschke, 2007; Asberg, Bowers, Renk, & McKinney, 2008; Verschoor & Markus, 2011; Credé & Niehorster, 2012). Assessment of stress is an important component of efforts to assist students in benefiting from the college experience, and instruments used for such assessment should demonstrate reliability and construct validity. The College Student Stress Scale was developed to provide a brief and global measure of college stress that includes appraisal of the ability to maintain control (Feldt, 2008). A recent study confirmed that the College Student Stress Scale has a two-factor structure; however, use of a threeitem subscale for the second factor was not recommended due to low reliability (Feldt & Koch, 2011). Structural and external aspects of construct validity (Messick, 1995) were established in two previous studies (Feldt, 2008; Feldt & Koch, 2011). One question that remains is the extent to which the College Student Stress Scale is invariant across gender, the generalizability aspect of construct validity (Messick, 1995). Invariance testing at construct and item levels provides tests of generalizability across populations, and multigroup confirmatory factor analysis (CFA) is commonly used for such testing (Dimitrov, 2010; Sass, 2011). Comparison of gender means based on observed scores assumes measurement invariance, and results of such comparisons when the assumption is violated may be erroneous (Sass, 2011). The purpose of the present study was to examine measurement invariance across gender of the seven-item subscale.
Address correspondence to Ronald C. Feldt, Department of Psychology, Social Work, Sociology, and International Studies, Mount Mercy University, Cedar Rapids, IA 52402 or e-mail ([email protected]). 1
DOI 10.2466/03.PR0.113x23z0
09-PR_Feldt_130064.indd 486
ISSN 0033-2941
19/11/13 4:24 PM
COLLEGE STUDENT STRESS SCALE
487
Method Participants included 460 undergraduate college students (353 women and 107 men; 365 first-year students and more than 92% Euro-American) who were enrolled in psychology classes. Age ranged from 17 to 44 years (M = 19.5 yr.; SD = 3.3). No gender difference in age was observed (p > .05). The College Student Stress Scale has 11 items and a Likert-type scale with response categories ranging from 1: Never to 5: Very often. It was administered in a classroom setting. Results and Discussion Data for this analysis were combined from two previous studies (Feldt, 2008; Feldt & Koch, 2011) to create a large enough sample to support an invariance analysis. Multigroup confirmatory factor analysis was used with maximum likelihood estimation. A principal component analysis was used to identify a reference indicator based on the most similar component loadings across gender (Finch & French, 2008). This study created a well-fitting model and determined goodness of fit for each gender, in addition to a combined configural model. A correlated error for Items 7 and 8 was estimated (Feldt & Koch, 2011). Fit indexes included the comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR). The chi-squared test was not used to assess goodness of fit because of its tendency to be influenced by sample size and to reject hypothesized models when negligible differences exist (Brown, 2006; West, Taylor, & Wu, 2012). A series of constrained models was created with equality constraints on factor loadings, item intercepts, and factor variance. Change in CFI (< –.01) (Cheung & Rensvold, 2002) was used for model comparison because it is less likely than the chi-squared difference test (Δχ2) to result in Type I errors when models do not have perfect fit (French & Finch, 2011). Good fit was indicated by CFI values > .95 and SRMR values < .05 for the configural, baseline model for each gender and the combined model (Table 1). Standardized regression weights ranged from .42 to .80 for women and from .58 to .86 for men. Results of a series of comparisons of constrained models indicated invariance of the correlated error estimate, factor loadings, and factor variance. In contrast, the intercept for Item 10 was observed to be noninvariant (ΔCFI = –.018), thus indicating differential item functioning (DIF). Improved fit was observed when the intercept was freely estimated. Item 10 intercept estimates for women and men were 1.79 and 2.08, respectively. This indicates that men are more likely to endorse this item (“no longer in control”). All other intercepts were noninvariant. The difference between gender latent means was not statistically significant whether the DIF item was included or was excluded from the analysis,
09-PR_Feldt_130064.indd 487
19/11/13 4:24 PM
488
R. C. FELDT & C. UPDEGRAFF TABLE 1 MEASUREMENT INVARIANCE OF THE COLLEGE STUDENT STRESS SCALE ACROSS GENDER (N = 460) Model
χ2
df
CFI
RMSEA (90%CI)
SRMR
Women
51.18
13
.963 .091 (.066, .118)
.037
Men
20.67
13
.982 .075 (.000, .132)
.034
Combined
71.88
26
.968 .062 (.045, .079)
.037
Factor loadings equal
80.41
32
.966 .057 (.042, .073)
.040
Item intercepts equal
112.99
38
.948 .066 (.042, .073)
.040
Item 10 intercept estimated
100.94
37
.956 .061 (.052, .080)
.040
Factor variance equal 103.40 38 .954 .061 (.047, .076) .044 Note.—χ2 = chi-squared goodness-of-fit test, CFI = comparative fit index, RMSEA (with 90% confidence interval) = root mean square error of approximation, and SRMR = standardized root mean square residual.
with estimates of the latent variable mean difference of .15 and .22, respectively (both p > .05). In addition, comparison of the fully- (all intercepts constrained) and partially-invariant (intercept for Item 10 was estimated) models indicated a slight increase in the estimated difference to .22 in the partially-invariant model. However, both estimates were not statistically significant (p > .05). Our results support the partial measurement invariance of the seven-item measure of college stress with minimal impact of the noninvariant intercept for Item 10 (Byrne, Shavelson, & Muthén, 1989). The seven-item measure should serve as a global measure of college stress for researchers who wish to investigate relationships between global stress and other constructs. The measure should be particularly useful when researchers employ relatively lengthy instruments within a brief session. Although the second sample (Feldt & Koch, 2011) was slightly older than the first sample (Feldt, 2008) (p < .001), parameter estimates for the correlated error, factor loadings, intercepts, and variance of the latent variable were observed to be invariant across the two samples. In contrast, the estimated difference in latent means was greater for the older sample (p < .05). The major limitation includes limited generalizability due to the fact that the sample was comprised primarily of Euro-American women in their first-year of college and Euro-American men. In addition, the sample consisted of students committed to service-oriented majors (nursing, psychology, and social work). Subsequent research should include additional populations that vary in race and ethnicity, in addition to year in college and college major.
09-PR_Feldt_130064.indd 488
19/11/13 4:24 PM
COLLEGE STUDENT STRESS SCALE
489
REFERENCES
ASBERG, K. K., BOWERS, C., RENK, K., & MCKINNEY, C. (2008) A structural equation modeling approach to the study of stress and psychological adjustment in emerging adults. Child Psychiatry and Human Development, 39, 481-501. DOI: 10.1007/s10578008-0102-0 BROWN, T. A. (2006) Confirmatory factor analysis for applied research. New York: Guilford. BYRNE, B. M., SHAVELSON, R. J., & MUTHÉN, B. (1989) Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychological Bulletin, 105, 456-466. DOI: 10.1037/0033-2909.105.3.456 CHEUNG, G. W., & RENSVOLD, R. B. (2002) Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255. DOI: 10.1207/ S15328007SEM0902_5 CREDÉ, M., & NIEHORSTER, S. (2012) Adjustment to college as measured by the Student Adaptation to College Questionnaire: a quantitative review of its structure and relationships with correlates and consequences. Educational Psychology Review, 24, 133-165. DOI: 10.1007/s10648-011-9184-5 DIMITROV, D. M. (2010) Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43, 121-149. DOI: 10.1177/0748175610373459 FELDT, R. C. (2008) Development of a brief measure of college stress: the College Student Stress Scale. Psychological Reports, 102, 855-860. DOI: 10.2466/pr0.102.3.855-860 FELDT, R. C., & KOCH, C. (2011) Reliability and construct validity of the College Student Stress Scale. Psychological Reports, 108, 660-666. DOI: 10.2466/02.08.13.16.PR0.108.2. 660-666 FINCH, W. H., & FRENCH, B. F. (2008) Using exploratory factor analysis for locating invariant referents in factor invariance studies. Journal of Modern Applied Statistical Methods, 7, 223-233. Retrieved at http://www.jmasm.com/ FRENCH, B. F., & FINCH, W. (2011) Model misspecification and invariance testing using confirmatory factor analytic procedures. Journal of Experimental Education, 79, 404428. DOI: 10.1080/00220973.2010.517811 MESSICK, S. (1995) Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749. DOI: 10.1037/0003-066X.50.9.741 SASS, D. A. (2011) Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment, 29, 347-363. DOI: 10.1177/0734282911406661 SEGRIN, C., HANZAL, A., DONNERSTEIN, C., TAYLOR, M., & DOMSCHKE, T. J. (2007) Social skills, psychological well-being, and the mediating role of perceived stress. Anxiety, Stress & Coping: An International Journal, 20, 321-329. DOI: 10.1080/10615800701282252 VERSCHOOR, E., & MARKUS, C. (2011) Effects of acute psychosocial stress exposure on endocrine and affective reactivity in college students differing in the 5-HTTLPR genotype and trait neuroticism. Stress: The International Journal on the Biology of Stress, 14, 407-419. DOI: 10.3109/10253890.2010.548886 WEST, S. G., TAYLOR, A. B., & WU, W. (2012) Model fit and model selection in structural equation modeling. In R. Hoyle (Ed.), Handbook of structural equation modeling. New York: Guilford. Pp. 209-231. Accepted September 17, 2013.
09-PR_Feldt_130064.indd 489
19/11/13 4:24 PM
Copyright of Psychological Reports is the property of Ammons Scientific, Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
GENDER INVARIANCE OF THE COLLEGE STUDENT STRESS SCALE1 RONALD C. FELDT AND CHRISTINA UPDEGRAFF Mount Mercy University Summary.—Assessment of perceived stress may be an important prerequisite to deployment of effective coping in efforts to help college students adjust to academic and social demands of college. The study examined the extent to which a seven-item measure of the College Student Stress Scale is invariant across gender. Results indicated invariance of factor loadings, factor variance, and all but one item intercept. No statistically significant gender difference was observed between latent variable means.
The inability to manage the stress response in college is associated with poor college adjustment, depression, and reduced life satisfaction (Segrin, Hanzal, Donnerstein, Taylor, & Domschke, 2007; Asberg, Bowers, Renk, & McKinney, 2008; Verschoor & Markus, 2011; Credé & Niehorster, 2012). Assessment of stress is an important component of efforts to assist students in benefiting from the college experience, and instruments used for such assessment should demonstrate reliability and construct validity. The College Student Stress Scale was developed to provide a brief and global measure of college stress that includes appraisal of the ability to maintain control (Feldt, 2008). A recent study confirmed that the College Student Stress Scale has a two-factor structure; however, use of a threeitem subscale for the second factor was not recommended due to low reliability (Feldt & Koch, 2011). Structural and external aspects of construct validity (Messick, 1995) were established in two previous studies (Feldt, 2008; Feldt & Koch, 2011). One question that remains is the extent to which the College Student Stress Scale is invariant across gender, the generalizability aspect of construct validity (Messick, 1995). Invariance testing at construct and item levels provides tests of generalizability across populations, and multigroup confirmatory factor analysis (CFA) is commonly used for such testing (Dimitrov, 2010; Sass, 2011). Comparison of gender means based on observed scores assumes measurement invariance, and results of such comparisons when the assumption is violated may be erroneous (Sass, 2011). The purpose of the present study was to examine measurement invariance across gender of the seven-item subscale.
Address correspondence to Ronald C. Feldt, Department of Psychology, Social Work, Sociology, and International Studies, Mount Mercy University, Cedar Rapids, IA 52402 or e-mail ([email protected]). 1
DOI 10.2466/03.PR0.113x23z0
09-PR_Feldt_130064.indd 486
ISSN 0033-2941
19/11/13 4:24 PM
COLLEGE STUDENT STRESS SCALE
487
Method Participants included 460 undergraduate college students (353 women and 107 men; 365 first-year students and more than 92% Euro-American) who were enrolled in psychology classes. Age ranged from 17 to 44 years (M = 19.5 yr.; SD = 3.3). No gender difference in age was observed (p > .05). The College Student Stress Scale has 11 items and a Likert-type scale with response categories ranging from 1: Never to 5: Very often. It was administered in a classroom setting. Results and Discussion Data for this analysis were combined from two previous studies (Feldt, 2008; Feldt & Koch, 2011) to create a large enough sample to support an invariance analysis. Multigroup confirmatory factor analysis was used with maximum likelihood estimation. A principal component analysis was used to identify a reference indicator based on the most similar component loadings across gender (Finch & French, 2008). This study created a well-fitting model and determined goodness of fit for each gender, in addition to a combined configural model. A correlated error for Items 7 and 8 was estimated (Feldt & Koch, 2011). Fit indexes included the comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR). The chi-squared test was not used to assess goodness of fit because of its tendency to be influenced by sample size and to reject hypothesized models when negligible differences exist (Brown, 2006; West, Taylor, & Wu, 2012). A series of constrained models was created with equality constraints on factor loadings, item intercepts, and factor variance. Change in CFI (< –.01) (Cheung & Rensvold, 2002) was used for model comparison because it is less likely than the chi-squared difference test (Δχ2) to result in Type I errors when models do not have perfect fit (French & Finch, 2011). Good fit was indicated by CFI values > .95 and SRMR values < .05 for the configural, baseline model for each gender and the combined model (Table 1). Standardized regression weights ranged from .42 to .80 for women and from .58 to .86 for men. Results of a series of comparisons of constrained models indicated invariance of the correlated error estimate, factor loadings, and factor variance. In contrast, the intercept for Item 10 was observed to be noninvariant (ΔCFI = –.018), thus indicating differential item functioning (DIF). Improved fit was observed when the intercept was freely estimated. Item 10 intercept estimates for women and men were 1.79 and 2.08, respectively. This indicates that men are more likely to endorse this item (“no longer in control”). All other intercepts were noninvariant. The difference between gender latent means was not statistically significant whether the DIF item was included or was excluded from the analysis,
09-PR_Feldt_130064.indd 487
19/11/13 4:24 PM
488
R. C. FELDT & C. UPDEGRAFF TABLE 1 MEASUREMENT INVARIANCE OF THE COLLEGE STUDENT STRESS SCALE ACROSS GENDER (N = 460) Model
χ2
df
CFI
RMSEA (90%CI)
SRMR
Women
51.18
13
.963 .091 (.066, .118)
.037
Men
20.67
13
.982 .075 (.000, .132)
.034
Combined
71.88
26
.968 .062 (.045, .079)
.037
Factor loadings equal
80.41
32
.966 .057 (.042, .073)
.040
Item intercepts equal
112.99
38
.948 .066 (.042, .073)
.040
Item 10 intercept estimated
100.94
37
.956 .061 (.052, .080)
.040
Factor variance equal 103.40 38 .954 .061 (.047, .076) .044 Note.—χ2 = chi-squared goodness-of-fit test, CFI = comparative fit index, RMSEA (with 90% confidence interval) = root mean square error of approximation, and SRMR = standardized root mean square residual.
with estimates of the latent variable mean difference of .15 and .22, respectively (both p > .05). In addition, comparison of the fully- (all intercepts constrained) and partially-invariant (intercept for Item 10 was estimated) models indicated a slight increase in the estimated difference to .22 in the partially-invariant model. However, both estimates were not statistically significant (p > .05). Our results support the partial measurement invariance of the seven-item measure of college stress with minimal impact of the noninvariant intercept for Item 10 (Byrne, Shavelson, & Muthén, 1989). The seven-item measure should serve as a global measure of college stress for researchers who wish to investigate relationships between global stress and other constructs. The measure should be particularly useful when researchers employ relatively lengthy instruments within a brief session. Although the second sample (Feldt & Koch, 2011) was slightly older than the first sample (Feldt, 2008) (p < .001), parameter estimates for the correlated error, factor loadings, intercepts, and variance of the latent variable were observed to be invariant across the two samples. In contrast, the estimated difference in latent means was greater for the older sample (p < .05). The major limitation includes limited generalizability due to the fact that the sample was comprised primarily of Euro-American women in their first-year of college and Euro-American men. In addition, the sample consisted of students committed to service-oriented majors (nursing, psychology, and social work). Subsequent research should include additional populations that vary in race and ethnicity, in addition to year in college and college major.
09-PR_Feldt_130064.indd 488
19/11/13 4:24 PM
COLLEGE STUDENT STRESS SCALE
489
REFERENCES
ASBERG, K. K., BOWERS, C., RENK, K., & MCKINNEY, C. (2008) A structural equation modeling approach to the study of stress and psychological adjustment in emerging adults. Child Psychiatry and Human Development, 39, 481-501. DOI: 10.1007/s10578008-0102-0 BROWN, T. A. (2006) Confirmatory factor analysis for applied research. New York: Guilford. BYRNE, B. M., SHAVELSON, R. J., & MUTHÉN, B. (1989) Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychological Bulletin, 105, 456-466. DOI: 10.1037/0033-2909.105.3.456 CHEUNG, G. W., & RENSVOLD, R. B. (2002) Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255. DOI: 10.1207/ S15328007SEM0902_5 CREDÉ, M., & NIEHORSTER, S. (2012) Adjustment to college as measured by the Student Adaptation to College Questionnaire: a quantitative review of its structure and relationships with correlates and consequences. Educational Psychology Review, 24, 133-165. DOI: 10.1007/s10648-011-9184-5 DIMITROV, D. M. (2010) Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43, 121-149. DOI: 10.1177/0748175610373459 FELDT, R. C. (2008) Development of a brief measure of college stress: the College Student Stress Scale. Psychological Reports, 102, 855-860. DOI: 10.2466/pr0.102.3.855-860 FELDT, R. C., & KOCH, C. (2011) Reliability and construct validity of the College Student Stress Scale. Psychological Reports, 108, 660-666. DOI: 10.2466/02.08.13.16.PR0.108.2. 660-666 FINCH, W. H., & FRENCH, B. F. (2008) Using exploratory factor analysis for locating invariant referents in factor invariance studies. Journal of Modern Applied Statistical Methods, 7, 223-233. Retrieved at http://www.jmasm.com/ FRENCH, B. F., & FINCH, W. (2011) Model misspecification and invariance testing using confirmatory factor analytic procedures. Journal of Experimental Education, 79, 404428. DOI: 10.1080/00220973.2010.517811 MESSICK, S. (1995) Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749. DOI: 10.1037/0003-066X.50.9.741 SASS, D. A. (2011) Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment, 29, 347-363. DOI: 10.1177/0734282911406661 SEGRIN, C., HANZAL, A., DONNERSTEIN, C., TAYLOR, M., & DOMSCHKE, T. J. (2007) Social skills, psychological well-being, and the mediating role of perceived stress. Anxiety, Stress & Coping: An International Journal, 20, 321-329. DOI: 10.1080/10615800701282252 VERSCHOOR, E., & MARKUS, C. (2011) Effects of acute psychosocial stress exposure on endocrine and affective reactivity in college students differing in the 5-HTTLPR genotype and trait neuroticism. Stress: The International Journal on the Biology of Stress, 14, 407-419. DOI: 10.3109/10253890.2010.548886 WEST, S. G., TAYLOR, A. B., & WU, W. (2012) Model fit and model selection in structural equation modeling. In R. Hoyle (Ed.), Handbook of structural equation modeling. New York: Guilford. Pp. 209-231. Accepted September 17, 2013.
09-PR_Feldt_130064.indd 489
19/11/13 4:24 PM
Copyright of Psychological Reports is the property of Ammons Scientific, Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
