RESEARCH ARTICLE

Developing a Comprehensive School Connectedness Scale for Program Evaluation JANE J. CHUNG-DO, DrPHa DEBORAH A. GOEBERT, DrPHb JANICE Y. CHANG, PsyDc FUMIAKI HAMAGANI, PhDd

ABSTRACT BACKGROUND: Evidence show that school connectedness is important to youth wellness. However, considerable inconsistency in the concepts and measures of school connectedness exists across studies. In addition, many measures do not capture the multifaceted dimensions of the school connectedness construct. This study examined the psychometric properties of a school connectedness scale that aimed to measure comprehensively the key constructs of school connectedness. METHODS: The scale was developed with teachers and tested with an ethnically diverse sample of 717 high school students enrolled in a school connectedness course using confirmatory factor analysis. RESULTS: Confirmatory factor analyses demonstrated the association of the 15 items with the 5 factors identified in the literature—school involvement, academic motivation, school attachment, teacher support, and peer relations (χ 2 = 439.99, df = 83, p < .0001, Comparative Fit Index = 0.991, Tucker-Lewis index = 0.988, root mean square error of approximation = 0.077). Cronbach coefficient alphas for the factors ranged from 0.73 to 0.93. CONCLUSIONS: Although further tests need to be conducted to assess its validity and reliability, this newly developed scale may provide researchers a tool to measure comprehensively school connectedness for program evaluation. Keywords: child and adolescent health; curriculum; evaluation; Asians and Pacific Islanders; Hawaii. Citation: Chung-Do JJ, Goebert DA, Chang JY, Hamagani F. Developing a comprehensive school connectedness scale for program evaluation. J Sch Health. 2015; 85: 179-188. Received on May 1, 2013 Accepted on October 22, 2014

H

igh rates of school failure, substance use, violence, pregnancy, and sexually transmitted diseases among our youth have increased the urgency of researchers to identify risk and protective factors to create effective programs.1-3 Schools are an ideal location to implement youth interventions because they provide access to large numbers of youth. Youth spend the majority of their day in school, where their identities and values are often shaped. Studies have found that those who feel connected to their schools are more likely to have positive educational and health outcomes. Resnick et al4

was one of the first research teams to define and measure school connectedness through the National Longitudinal Study of Adolescent Health (Add Health Survey). This seminal study found that students’ sense of school connectedness was one of the strongest protective factors of youth high-risk behaviors, such as substance use, violence, and suicidality. Since then, school connectedness has been gaining recognition as an important protective factor for positive youth development. In 2003, the Centers for Disease Control and Prevention (CDC) Division of Adolescent and School Health and The Johnson Foundation sponsored the Wingspread Conference that brought together key

a Assistant Professor, ([email protected]), Department of Public Health Sciences, University of Hawai’i at M¯anoa, 1960 East-West Rd, Biomedical Building, D104D Honolulu, HI 96822. bProfessor, ([email protected]), Department of Psychiatry, University of Hawai’i at M¯ anoa, John A. Burns School of Medicine, 1356 Lusitana St., 4th Floor, Honolulu, HI 96813. c Affiliate Professor, ([email protected]), Department of Psychiatry, University of Hawai’i at M¯ anoa, John A. Burns School of Medicine, 677 Ala Moana Blvd., Suite 301, Honolulu, HI 96813. dAffiliate Professor, ([email protected]), Department of Psychiatry, University of Hawai’i at M¯ anoa, John A. Burns School of Medicine, 677 Ala Moana Blvd., Suite 301, Honolulu, HI 96813.

Address correspondence to: Jane J. Chung-Do, Assistant Professor, ([email protected]), Department of Public Health Sciences, University of Hawai’i at M¯anoa, 1960 East-West Rd, Biomedical Building, D104D Honolulu, HI 96822. This manuscript was supported in part by Grant R49/CCR918619-01 and Cooperative Agreement #1 U49/CE000749-01 from the Centers for Disease Control and Prevention (CDC). The contents of this manuscript are solely the responsibility of the author and do not necessarily reflect the official views of the funding agency. The authors thank Kailua High School PTP/L team for their partnership and the staff of the Asian/Pacific Islander Youth Violence Prevention Center for their assistance.

Journal of School Health



March 2015, Vol. 85, No. 3



© 2015, American School Health Association • 179

educational and health researchers and representatives to review existing studies on school connectedness and make recommendations on strategies to promote school connectedness.5 Youth who feel a strong sense of connection to their schools tend to have high academic achievement and lower rates of substance use.6-8 Jennings9 found that school involvement and supportive relationships were positively related to grade point averages (GPAs). Those who lack a strong sense of school connectedness often experience academic problems, which are considered to be a gateway to delinquency and risky behaviors.10 Those who participate in extracurricular activities are less likely to drop out from school and engage in criminal behaviors.11-13 Involvement in school activities that are meaningful, relevant, and interesting to students can enhance motivation for learning and identification with the school, which supports the development and attainment of educational goals. Youth who have caring relationships with others in the school, such as their teachers and peers, are also more likely to identify with the school as well as actively engage in their learning experience. Strong teacher support can also prevent and reduce the likelihood of youth experimenting with drugs, alcohol, and sexual activity.7 For example, teacher support can prevent the initiation of cigarette and marijuana use as well as the escalation from occasional to regular use. School connectedness can also have effects beyond the individual behaviors. Buckley, Sheehan, and Chapman14 found that school connectedness is more predictive than parent connectedness in students’ willingness to intervene if their friends are engaging in risky behaviors. Despite the growing research on school connectedness, the concepts and measures of school connectedness are inconsistently utilized and not well defined.15-17 Studies often use the term ‘‘school connectedness’’ synonymously and interchangeably with other terms, such as school engagement, school bonding, school attachment, school climate, school involvement, and school commitment.18-20 Although most researchers agree that the concept of school connectedness include behavioral, affective, and cognitive dimensions of youth development, 15,21,22 many definitions of school connectedness only capture the affective aspects of school connectedness. For example, the CDC5 defines school connectedness as students’ perceptions of how much adults in the school care about them personally as well as their learning. Bonny et al1 provide a similar definition by conceptualizing school connectedness as the feeling of closeness to school staff and the school environment. Whitlock defines school connectedness as a ‘‘psychological state of belonging in which individual youth perceive that they and other youth are cared for, trusted, and respected by collections 180 •

Journal of School Health



March 2015, Vol. 85, No. 3 •

of adults that they believe hold the power to make institutional and policy decisions.’’17(p15) A review of 25 school connectedness studies conducted by Johnson16 revealed that various measures of school connectedness were used across the studies, which included peer relations, teacher support, academic success, classroom and school culture, perceptions of safety, and physical disorder. Libbey’s18 review found a similar pattern noting that school connectedness is often measured by different scales with different names within the same data set. Moreover, some studies use a single question to measure school connectedness, such as ‘‘How do you feel about your school?’’23 One of the most widely used scales is the School Connectedness Scale that was adapted from the Add Health Survey.4 Although this scale has been shown to have good internal consistency and validity,24 it is also utilized inconsistently across studies. The original scale in the Add Health Survey contained 6 items, whereas other studies have used anywhere from 5 to 8 items in their scale.1,25-27 Although it is widely used, this scale measures only affective components of school connectedness, such as feelings of belonging to the school, without assessing the behavioral and cognitive components.18,19,24 Given the importance of the role of school connectedness on youth wellness, accurately identifying and measuring school connectedness is imperative. To develop a scale that captures the multidimensionality of the school connectedness construct, this study aimed to examine the psychometric properties of a newly developed comprehensive school connectedness scale that was used as part of high school course evaluation.

METHODS Participants Recognizing the importance of school connectedness, Kailua High School (KHS), created the Personal Transition Plan/Leadership (PTP/L) course in 2007, which is designed to build students’ sense of school connectedness in developmentally appropriate ways from freshman to senior year. Kailua High School is a public high school in Hawai’i with an ethnically diverse student population. All KHS students are required to register and participate in the weekly PTP/L course every year. The class sizes are small with about 10-13 students in each class to help students develop a positive relationship with at least one adult at the school. Students remain with the same PTP/L teacher and classmates throughout their time at KHS to strengthen these relationships.28 All KHS students who were present on the day of the survey administration were invited to participate in the survey.

© 2015, American School Health Association

Instrument A 3-page survey was developed to evaluate the course by a team of KHS teachers and university staff members. A literature review was initially conducted to gather items from various existing scales.29-31 Approximately 17 items were identified and adapted from existing scales to address the 3 components of school connectedness as posited by Jimerson et al.22 The survey also included 25 questions related to students’ perceptions of the course and the knowledge gained throughout the course, which were not included in the analyses for this study. A total of 5 subscales were created based on the content validity of the 17 school connectedness items. These subscales included school involvement (3 items), academic motivation (3 items), school attachment (3 items), teacher support (5 items), and peer relations (3 items) (Table 1). The items measuring school involvement were adapted from Jenkin’s29 measure of school involvement. The names of the activities were adapted to reflect the activities occurring specifically at the school. The items measuring academic motivation were adapted from Jenkin’s29 measure of school commitment. The items measuring school attachment were adapted from McNeely, Nonnemaker, and Blum’s30 measure of school connectedness while teacher support and peer relations were adapted from the School Bonding Index Revised.31 All quantitative items on the survey were measured with a 5-point Likert scale with 1 being strongly disagree and 5 being strongly agree. Five demographic questions were also asked of the students: (1) What is your grade level in school right now? (2) What is your sex? (3) On average, what were your grades on your last report card? (4) Which of the following [ethnicity] do you most strongly identify with? and (5) Do you qualify to get free or reduced-cost lunch? Sex was coded as male = 0, female = 1. Ethnicity was grouped into 6 racial/ethnic categories: Native Hawaiian—defined as indigenous people of Hawai’i; Pacific Islander—defined as immigrant and migrant from the Pacific Islands including Samoans, Tongans, and Micronesians; Filipino; Japanese; and Caucasian. As an indicator for socioeconomic status, students who qualify for free or reduced-cost lunch were coded as 1, and those who do not qualify were coded as 0. Selfreported GPA was calculated as a continuous variable from 1.3, which was the lowest end of the range, to 4.0. Grade level was determined by school records and coded as a categorical variable. Procedures The survey was distributed and administered on the last day of the school year in spring 2010. Students were informed that the survey is voluntary and confidential, and the results of the survey will not Journal of School Health



affect their grades. Each student’s survey contained an ID code to protect confidentiality. Students were instructed to place their completed surveys in a common manila envelope. Because seniors ended their school year earlier than the rest of the lower grades, seniors received the survey 1 week before the rest of the students. The survey took approximately 30 minutes to complete. Data Analysis Surveys were electronically scanned using Teleform 10.2 software package (TELEform Elite; Cardiff Software Ltd, Vista, CA), and the data were transported into SAS 9.2 (2010; SAS Institute Inc., Cary, NC) for cleaning and coding. For the 17 items of interest, the data were coded so higher scores reflect a stronger sense of school connectedness. Data were first examined by comparing the correlations among the 17 school connectedness items. Items that possessed weak correlations, defined as r = .3 or less, were noted. The data were further analyzed using Mplus 4.232 to conduct confirmatory factor analyses to test the fit of the proposed factor structure (Figure 1). To account for data skewness, variables were treated as categorical variables and several correlated uniqueness factors between items were allowed in the model.33,34 Confirmatory factor analyses with ordered categorical variables that Muthen pioneered were utilized.35,36 The typical estimation technique for categorical structural equation modeling is weighted least square (WLS). Considering sufficient sample size of our study and conservative estimation based on arguments by Muthen, du Toit, and Spisic37 and Nussbeck et al,38 we employed WLS rather than robust Weighted Least Squares Means and Variance adjusted (WLSMV) estimators. WLSMV uses the diagonal of the weight matrix in the estimation, which can facilitate parameter estimation regardless of the small sample size, whereas WLS uses the full weight matrix, which does not allow robust computation of weight matrix with small sample sizes. Weighted least square and WLSMV use the full weight matrix to compute standard errors and chi-square. Neither WLS nor WLSMV estimator uses a fitting function that attempts to minimize the residuals. Because WLSMV only uses the diagonal weight matrix to get the estimates, the residuals of outcome variables tend to be closer to zero than using the WLS estimator. Items were removed from the model if the factor loadings were lower than 0.50 in the 1-factor solution. Indicators with low factor loadings were excluded to avoid the risk of lowering internal consistency of the latent construct.39 Cases with missing data were imputed using a full information estimation approach. Because the variables were categorical, WLS was used with pairwise deletion. To test the factor structures of the scale, a 1-factor model was tested, which

March 2015, Vol. 85, No. 3



© 2015, American School Health Association • 181

Table 1. Description of School Connectedness Scale Jimerson et al’s (2003) Framework Behavioral

Cognitive

Affective

Factors in School Connectedness Scale

Item

School involvement, INVOLVE

1 2 3

Statements

Activities Events Meet people

Academic motivation, ACADEMIC 4 5 6 School attachment, ATTACH 7 8 9 Teacher support, TEACHER 10 11 12 13 14 Peer relations, PEER 15 16 17

This course has encouraged me to get involved in school-related activities This course has encouraged me to attend school-related events This course has helped me get to know people at the school that I may not have otherwise met or gotten to know Grades This course has helped me understand the importance of my grades Classes This course has motivated me to do better in my classes Education This course had helped me understand the importance of education Part of school This course has contributed to making me feel like I am part of this school Unhappy This course has made me feel unhappy to be at school. (reverse-coded) Give to school This course has made me want to give back to our school Teacher I feel connected with my teacher Advice I can talk to my teacher if I have a problem or need advice Lessons My teacher makes the lessons interesting and meaningful Unfair My teacher treats me unfairly (reverse-coded) Expectations My teacher has high expectations of me Student connect I feel connected with the students in my class Listen Students in my class respectfully listen to each other during class discussions Get along This course has helped me to get along with others

Figure 1. Standardized Coefficients of 5-Factor Model of the School Connectedness Scale 4 2 1

.94

6

.98

.96

8*

.96

.89

.92

.92

5

7

3

Academic motivation

Student involvement

9

School attachment

.85

.95 .97

.97

School connectedness .93

Teacher support

.96

10 .93

11

.93

.77

.87

12

Peer relations

14

13*

.83

15

.99 .84

17 16

χ2 = 439.99, df = 83, p < .0001, CFI = 0.991, TLI = 0.988, RMSEA = 0.077 *Items in shaded boxes were removed from the final model

assumed that the construct of school connectedness is 1 dimensional. A 3-factor model was tested to assess if the factors fit into Jimerson et al’s22 framework. A 5-factor model was also tested to assess if the factors fit into the original components of school connectedness as adapted from other studies. All models were first tested as a first-order model to confirm that all factors were distinct separate constructs. To test the 182



Journal of School Health



March 2015, Vol. 85, No. 3



theory that these factors are dimensions of the school connectedness construct, models were further tested as a second-order model. To test the validity of the imputation assumptions,40 goodness-of-fit indices (GFI) of the imputed data were compared with complete cases. Several GFI were used to assess overall model fit. The Comparative Fit Index (CFI) was used to compare the ability of a model © 2015, American School Health Association

to replicate the variance-covariance matrix compared to no model at all.41 The CFI values range from 0 to 1, with CFI values greater than 0.90 for models that are considered a good fit. The Tucker-Lewis index (TLI) is a non-normed fit index that allows for penalty for adding parameters. Similar to CFI, TLI values of greater than 0.90 are considered to indicate a good fit.42 The root mean square error of approximation (RMSEA) is a measure of discrepancy of the model to population covariance matrix per degree of freedom.43,44 Thus, RMSEA was regarded as a badness of fit to the asymptotic population covariance. A well-fitting model should have an RMSEA value that approaches zero (perfect approximation), although a value of 0.08 or less indicates a reasonable fit.43-46 Cronbach alpha was examined for each of the 5 factors in the final model to assess internal consistency. Evidence for concurrent validity was determined by examining the mean correlations among the 5 factors.

RESULTS Sample Description Of 916 students, 749 (82%) students completed the survey. A total of 717 usable surveys were used in the following analyses. Approximately 52% (366) of the study participants were boys and 46% (325) identified themselves as Native Hawaiian, 14% (100) as Filipino, 13% (94) as Caucasian, 11% (76) as Japanese/Okinawan, 7% (47) as Pacific Islanders, with other ethnicities each accounting for less than 3% of the sample. Of 602 students who answered the question, half (50%) of the participants reported that they qualify to receive free or reduced-price school lunch. The mean of the self-reported GPA of the sample was 2.92 (1.3-4.0, SD = 0.739). The sample included 157 (22%) 9th graders, 190 (27%) 10th graders, 182 (25%) 11th graders, and 188 (26%) 12th graders. Bivariate Analysis Pearson correlations among the 17 school connectedness items revealed that feeling happy to be at school and being treated fairly by teacher had the weakest correlations across all items with r < .03. The correlations among the rest of the items were moderately to highly correlated with correlation coefficients ranging from r = .36 to .83. All correlations were positive, which suggests that the more students are involved in schoolrelated activities and events, the more likely they will be academically motivated. Similarly, students who feel a strong sense of attachment to their school are more connected to their teachers and classmates. Confirmatory Factor Analysis Various models were tested with confirmatory factor analyses to understand the psychometric structures Journal of School Health



of the school connectedness construct. The 2 items related to feeling happy to be at school and being treated fairly by teacher were removed from the final models that were tested due to these items being weakly correlated to almost all other items and having significantly lower factor loadings than all other items with less than 0.50.39 A 1-factor model of school connectedness was tested, which grouped all the constructs into 1 factor. This model had an inadequate RMSEA (χ 2 = 777.73, df = 90, p < .0001, CFI = 0.982, TLI = 0.979, RMSEA = 0.103). A 3-factor model was tested as a first-order, grouped by the 3 components posited by Jimerson et al’s framework (χ 2 = 647.80, df = 86, p < .0001, CFI = 0.985, TLI = 0.981, RMSEA = 0.095).22 The construct of school involvement was categorized as behavioral, academic motivation as cognitive, and school attachment, teacher support, and peer relations as affective. It was also tested as a second-order model by including the latent construct of school connectedness to assess if these 3 components contributed to the construct of school connectedness, which did not change the fit. A 5-factor first-order model provided a better fit (χ 2 = 368.75, df = 77, p < .0001, CFI = 0.992, TLI = 0.989, RMSEA = 0.073), which included the 5 constructs of school involvement, academic motivation, school attachment, teacher support, and peer relations as 5 separate factors. This 5-factor model was further tested as a second-order model (χ 2 = 439.99, df = 83, p < .0001, CFI = 0.991, TLI = 0.988, RMSEA = 0.077), which also provided a good fit. Given that the secondorder 5-factor model is nested within the first-order 5-factor model, we examined if a fit difference of these models is statistically significant. We found that the second-order factor model does not fit as well as the first-order 5-factor model (χ 2 = 71.24, df = 6, p < .0000000000002). The structure coefficients, which were relatively high, are provided in Table 2. Figure 1 shows the standardized coefficients that each of the 5 factors contributed to this final model. Reliability and Validity The correlations among the 5 factors were in the moderate to high range (r = .59-.76). This suggests that school involvement, academic motivation, school attachment, teacher support, and peer relations are not mutually exclusive constructs, or are independent. Thus, they are considered as subconstructs for school connectedness, the primary construct. Estimates of the internal consistency of the school connectedness scale were calculated using the Cronbach coefficient alpha. All scales had good internal consistency with coefficient alphas ranging from α = 0.73-0.93 (Table 3).47 Face validity is not easily quantifiable because it refers to qualitative and subjective opinions of

March 2015, Vol. 85, No. 3 •

© 2015, American School Health Association • 183

Table 2. Structure Coefficients Matrix Between Factors and Items∗

Activities Events Meet people Grades Classes Education Part of school Give to school Teacher Advice Lessons Expectations Student connect Listen Get along

INVOLE

ACADEMIC

ATTACHMENT

TEACHER

PEER

SCHOOL CONNECTEDNESS

0.9120 0.9250 0.8840 0.8741 0.9019 0.8889 0.8996 0.8046 0.8681 0.8499 0.7899 0.7008 0.7391 0.7444 0.8872

0.8436 0.8556 0.8177 0.9450 0.9750 0.9610 0.8852 0.7918 0.8547 0.8368 0.7778 0.6900 0.7275 0.7328 0.8734

0.8573 0.8695 0.8310 0.8741 0.9019 0.8889 0.9570 0.8560 0.8681 0.8499 0.7899 0.7008 0.7391 0.7444 0.8872

0.8290 0.8408 0.8036 0.8458 0.8726 0.8601 0.8699 0.7781 0.9550 0.9350 0.8690 0.7710 0.7152 0.7204 0.8586

0.8190 0.8307 0.7938 0.8354 0.8619 0.8495 0.8594 0.7687 0.8299 0.8125 0.7552 0.6700 0.8230 0.8290 0.9880

0.8837 0.8963 0.8566 0.9015 0.9301 0.9168 0.9283 0.8303 0.8958 0.8770 0.8151 0.7232 0.7621 0.7677 0.9149

∗ Structure coefficient matrix = L × Rho; L = standardized factor loading (factor pattern); Rho = correlation among factors in the model.

Table 3. Correlation Matrix for 5 Factors of the Final School Connectedness Scale and Cronbach Alpha of Each Factor Factors School involvement Academic motivation School attachment Teacher support Peer relations

School Involvement

Academic Motivation

1 0.76 0.73 0.61 0.59

1 0.70 0.62 0.59

School Attachment

1 0.62 0.60

Teacher Support

Peer Relations

Cronbach Alpha (α)

1

0.84 0.93 0.73 0.85 0.86

1 0.67

All correlations were significant at p < .0001.

researchers about whether the test appears to measure what it claims to at face value. To evaluate face validity of our instrument, additional structural equation models were explored. In these models, we kept the factors as an overall SEM. However, we randomly assigned test indicators to each of 5 factors disregarding which subconstruct these items are intended to measure. Thus, the first factor can be formed by a school involvement item, an academic motivation item, and a teacher support item, while the second factor can be a peer relation item, a school attachment item and another teacher support item. In some alternative confirmatory factor models, we created a structure of 1 double, 3 triplets, and 1 quadruplet to match the final 5-factor solution. In the other Confirmatory Factor Models (CFMs), the model consists of 5 triplet structures. We carried out these models to understand what would happen if we completely disregard the face contents of subscales of the test so that a factor consists of mismatched items. We found that deliberate misspecification of latent constructs exacerbated model misfits to the extent that these alternative models were no longer acceptable. Namely, RMSEA for alternative misspecified models exceeded 0.1 (ie, RMSEA > 0.100). Results suggest that test items for the same subscale category appear to belong together within the latent construct, thus measuring the same construct. 184



Journal of School Health



March 2015, Vol. 85, No. 3



Furthermore, we performed resampling validation analyses. We randomly selected 25%, 50%, and 75% of the 717 participants and ran the second-order confirmatory factor model to respective resamples. We found that parameter estimates of the validation resamples were close to the full sample analysis. This suggests that sizes of factor loadings are consistent regardless of the sample sizes of random resampling from our sample. The details of numerical differences of parameters between the full participant sample and the random resample analyses are given in Table 4.

DISCUSSION The results of this study support the concept of school connectedness as a multidimensional construct. The factor analyses conducted in this study confirmed the association of the 15 items with the 5 hypothesized factors: school involvement, academic motivation, school attachment, teacher support, and peer relations. Many studies use scales that assess only one of these factors to measure what is labeled as school connectedness.15,17,18 The results suggest that these 5 factors are distinct separate dimensions of the school connectedness construct. Although the results of the chi-square difference test suggested that the fit was slightly better for the first-order model, the results of the second-order model also indicated a good fit, © 2015, American School Health Association

Table 4. Numerical Differences of Parameter Estimates of Second-Order Factor Solutions as a Function of Alternative Random Resampling of N = 717 Participants

First-order factor loading INVOL→ NQ6_INVOL INVOL→ NQ7_INVOL INVOL→ NQ8_INVOL ACAD→ NQ9_ACAD ACAD→ NQ10_ACAD ACAD→ NQ11_ACAD ATTAC→ NQ12_ATTA ATTAC→ NQ14_ATTA TEACH→ NQ15_TEAC TEACH→ NQ16_TEAC TEACH→ NQ17_TEAC TEACH→ NQ19_TEAC PEER→ NQ20_PEER PEER→ NQ21_PEER PEER→ NQ22_PEER Second-order factor loading CONN→ INVOL CONN→ ACAD CONN→ ATTAC CONN→ TEACH CONN→ PEER

Full (N = 717) ParameterEstimatesλfull

75% Random Sample λfull -λ75%

50% Random Sample λfull -λ50%

25% Random Sample λfull -λ25%

0.912 0.925 0.884 0.945 0.975 0.961 0.957 0.856 0.955 0.935 0.869 0.771 0.823 0.829 0.988

0.0670 0.0690 0.0750 0.0330 0.0140 0.0360 0.0820 0.1290 0.0530 0.0690 0.0200 0.0650 0.0810 0.0620 0.0220

0.0500 0.0810 0.0920 0.0320 0.0250 0.0270 0.0910 0.2060 0.0420 0.0520 0.0570 0.1370 0.1000 0.0770 0.0210

0.0160 0.0230 0.0450 0.0080 0.0160 0.0470 0.0600 0.1710 0.0280 0.0480 0.0260 0.0530 0.0380 −0.0220 −0.0070

0.969 0.954 0.970 0.938 0.926

0.0260 0.0670 −0.0230 0.1280 0.0860

0.0510 0.0370 −0.0270 0.1390 0.1060

0.0440 0.0300 −0.0770 0.1300 0.1310

which aligns with the theoretical conceptualization of school connectedness as a multidimensional construct. The results of this study also suggested that there are multiple factors under the 3 components proposed by Jimerson et al’s22 framework, which was highlighted by the better fit of the 5-factor model compared to the 3-factor model. This newly developed scale may provide researchers and school practitioners an instrument to measure more accurately and comprehensively school connectedness for program evaluation. Examining school connectedness as a multidimensional construct can help deepen the understanding of the complexity of youth’s experiences in school and better inform current and future interventions that promote positive youth development. A multidimensional perspective of youth development is crucial to overall youth wellness and provides a richer characterization of the school connectedness construct. How youth behave, feel, and think are intricately interrelated and cannot be studied as isolated processes.21 Much of the interactions among the psychological, behavioral, and cognitive processes can have cumulative impacts that may not occur with the existence of just one of these components. For example, Lee and Smith48 found that social support by itself did not relate to student learning. Likewise, academic motivation did not directly impact student learning. Instead, they found that the combination of academic motivation and social support was significantly related to student learning outcomes.48 Journal of School Health



Measuring school connectedness as a multidimensional construct can also help program developers better understanding which components can have the greatest impact on specific health and educational outcomes. Future studies should focus on examining how the various dimensions of school connectedness may have differential effects on youth outcomes. For example, McNeely and Falci7 found that students who reported high levels of teacher support were less likely to engage in risky health behaviors, such as substance use, suicidal attempts, sexual activity, and weapon-related violence. However, feeling part of school and enjoying school did not necessarily prevent students from initiating these risky behaviors. Teacher involvement is critical to students’ sense of school connectedness, which emphasizes the importance of the roles of teachers.49 Parental connectedness may also have interactional effects with school connectedness on youth outcomes.50 Therefore, future studies should examine school connectedness that includes the perspectives and influences of teachers and parents.

Limitations One of this study’s limitations is that the wording of the survey questions was constructed to be specific to the PTP/L course. Research has indicated that individual- and classroom-level factors are more influential in determining school climate.51 As such, course-specific perceptions may represent a more

March 2015, Vol. 85, No. 3 •

© 2015, American School Health Association • 185

accurate reflection of overall school connectedness.52 Some of the wording of the questions may need to be adapted to reflect the specific program being evaluated and the psychometric properties should be further tested. There is also the possibility that students who had earlier exposure to the program may have different responses from students with less exposure, which could have altered the psychometric properties of the scale. Students who are chronically absent or truant may not have been present on the day of the survey administration. Students who may have difficulty reading or maintaining attention for a long period of time may have been less likely to complete the survey. Therefore, the resulting sample may be biased toward students with higher levels of school connectedness. The findings of this study are also limited to students from one high school in a specific region of the United States. Therefore, this scale needs to be validated with youth in other schools and areas to conduct multigroup comparisons. However, the diversity of the student sample of this study is a promising start to test the psychometric properties of this scale. In addition, the number of the items in the scale is manageable, which minimizes the possibility of participant burnout, especially with youth. Additional tests of validity and reliability, such as test-retest, should be conducted to assess the stability of the school connectedness measure. To validate the scale further, the relationship between students’ school connectedness measured with this scale and their educational and health outcomes should be examined.

To be able to implement and strengthen efforts to promote school connectedness, comprehensive tools must be developed to evaluate the effectiveness. This multidimensional school connectedness scale provides an evaluative tool for school practitioners for future program and course evaluation efforts. Although many programs and courses are implemented in the school-setting, few are evaluated.57 Evaluating school connectedness interventions with a multidimensional comprehensive scale can help school administrators and classroom instructors identify specific areas of strengths and improvement and make direct and immediate program changes. For example, one of the results from this course evaluation showed that the area of peer relations was being rated relatively low by the students. This finding motivated KHS to incorporate more activities that promote interactions and positive relationship-building opportunities among the students. An additional strength of this scale is the participatory nature of its development. The university staff shared existing scales from the literature, which a team of classroom teachers helped adapt to fit the purpose of the PTP/L course evaluation. Thus, the school-university partnership ensured that the scale was relevant to the needs of the school, which may ensure sustainability of this evaluation tool. Currently, existing school connectedness measures do not include multidimensional aspects of school connectedness that may be essential in predicting youth outcomes. This study found that there are various and distinct components of school connectedness. Although more studies are needed to validate further this newly developed scale, this study provides preliminary evidence for its potential utility. The scale sheds light on the importance of holistic approaches to youth engagement and development. Fostering a sense of attachment to the school within an individual student is influenced by the interpersonal relationships that are created among adult school staff and classmates. As the school connectedness literature moves forward, researchers should aim to use comprehensive measures of school connectedness that capture the multifaceted nature of this construct. By recognizing and capturing the multiple aspects of school connectedness to evaluate programs, educational and developmental wellness of youth can be enhanced.

IMPLICATIONS FOR SCHOOL HEALTH Schools can provide a supportive environment for youth wellness.53,54 School connectedness is a modifiable factor that gives hope for building resiliency among youth. Given that school connectedness generally decreases as youth grow older, it is imperative to implement strategies to enhance school connectedness in high schools when levels of parental involvement drastically drop.55 As students move through from middle to high school, they must quickly adapt to dramatic increases in class and school sizes and changes in school policies, while preparing for their post-high school plans. Because high school is a critical transitional period for youth, focusing efforts toward promoting school connectedness during this time may be beneficial to promote youth wellness and educational outcomes. Kailua High School’s PTP/L course may be a promising approach to enhance student outcomes through the relationships built and opportunities created by the course.56 In the current environment where schools are struggling to meet national standards with limited resources, the PTP/L provides a concrete example of how academic space can be created to promote school connectedness. 186



Journal of School Health



March 2015, Vol. 85, No. 3

Human Subjects Approval Statement This study was approved by the University of Hawai’i Institutional Review Board.

REFERENCES 1. Bonny AE, Britto MT, Klostermann BK, Hornung RW, Slap GB. School disconnectedness: identifying adolescents at risk. Pediatrics. 2000;106:1017-1021. •

© 2015, American School Health Association

2. Dahlberg LL. Youth violence in the United States: major trends, risk factors, and prevention approaches. Am J Prev Med. 1998;14(4):259-272. 3. Youngblade LM, Theokas C, Schulenberg J, Curry L, Huang IC, Novak M. Risk and promotive factors in families, schools, and communities: a contextual model of positive youth development in adolescence. Pediatrics. 2007;119:S47-S53. 4. Resnick MD, Bearman PS, Blum RW, et al. Protecting adolescents from harm. Findings from the National Longitudinal Study on Adolescent Health. JAMA. 1997;278(10):823-832. 5. Centers for Disease Control and Prevention. School Connectedness: Strategies for Increasing Protective Factors Among Youth. Atlanta, GA: US Department of Health and Human Services; 2009. 6. Martin AJ, Dowson M. Interpersonal relationships, motivation, engagement, and achievement: yields for theory, current issues, and educational practice. Rev Educ Res. 2009;79:327-365. 7. McNeely CA, Falci C. School connectedness and the transition into and out of health risk behavior among adolescents: a comparison of social belonging and teacher support. J Sch Health. 2004;74(7):284-292. 8. Voelkl KE, Frone MR. Predictors of substance use at school among high school students. J Educ Psychol. 2000;92(3):583592. 9. Jennings G. An exploration of meaningful participation and caring relationships as contexts for school engagement. Calif School Psychol. 2003;8:43-52. 10. Catalano RF, Loeber R, McKinney KC. School and Community Interventions to Prevent Serious and Violent Offending. Washington, DC: Juvenile Justice Bulletin, Office of Juvenile Justice and Delinquent Prevention (Department of Justice); 1999. 11. Mahoney J. From companions to convictions: peer groups, school engagement, and development of criminality. Paper presented at the Biennial Meeting of the Society for Research on Child Development. Washington, DC; April 1997. 12. Mahoney JL, Cairns BD, Farmer TW. Promoting interpersonal competence and educational success through extracurricular activity participation. J Educ Psychol. 2003;95(2):409-418. 13. McNeal RB. Extracurricular activity participation and dropping out of high school. Sociol Educ. 1995;68:62-80. 14. Buckley L, Sheeba M, Chapman R. Adolescent protective behavior to reduce drug and alcohol use, alcohol-related harm and interpersonal violence. J Drug Educ. 2009;39(3):289-301. 15. Appleton JJ, Christenson SL, Furlong MJ. Student engagement with school: critical methodological issues of the construct. Psychol Sch. 2008;45(5):369-386. 16. Johnson SL. Improving the school environment to reduce school violence: a review of the literature. J Sch Health. 2009;79(10):451-465. 17. Whitlock JL. Youth perceptions of life at school: contextual correlates of school connectedness in adolescence. Appl Dev Sci. 2006;10(1):13-29. 18. Libbey HP. Measuring student relationships to school: attachment, bonding, connectedness, and engagement. J Sch Health. 2004;74(7):274-283. 19. Sharkey JD, You S, Schnoebelen K. Relations among school assets, individual resilience, and student engagement for youth grouped by level of family functioning. Psychol Schools. 2008;45(5):402-418. 20. Zullig KJ, Koopman TM, Patton JM, Ubbes VA. School climate: historical review, instrument development, and school assessment. J Psychoeduc Assess. 2010;28(2):139-152. 21. Fredericks J, Blumenfeld P, Paris A. School engagement: potential of the concept, and state of the evidence. Rev Educ Res. 2004;74(1):59-109. 22. Jimerson SR, Campos E, Greif JL. Toward an understanding of definitions and measures of school engagement and related terms. Calif School Psychol. 2003;8:7-27.

Journal of School Health



23. Thomas SP, Smith H. School connectedness, anger behaviors, and relationships of violent and nonviolent American youth. Perspect Psychiatr Care. 2004;40(4):135-148. 24. Anderman EM. School effects on psychological outcomes during adolescence. J Educ Psychol. 2002;94(4):795-809. 25. Jacobson KC, Rowe DC. Genetic and environmental influences on the relationships between family connectedness, school connectedness, and adolescent depressed mood: sex difference. Dev Psychol. 1999;35(4):926-939. 26. McNeely CA. Connection to school. In: Moore KA, Lippman LH, eds. What Do Children Need to Flourish? Conceptualizing and Measuring Indicators of Positive Development. New York, NY: Springer; 2005:289-303. 27. Ozer EJ. The impact of violence on urban adolescents: longitudinal effects of perceived school connection and family support. J Adolesc Res. 2005;20(2):167-192. 28. Black S. Together again: the practice of looping keeps students with the same teachers. Am School Board J. 2000;187(6): 40-43. 29. Jenkins P. School delinquency and the school social bond. J Res Crime Delinq. 1997;34(3):337-367. 30. McNeely CA, Nonnemaker JM, Blum RW. Promoting school connectedness: evidence from the National Longitudinal Study of Adolescent Health. J Sch Health. 2002;72(4):138-146. 31. Rodney LW, Johnson DL, Srivastava RP. The impact of culturally relevant violence prevention models on school-age youth. J Prim Prev. 2005;26(5):439-454. ´ LK, Muthen ´ BO. Mplus User’s Guide. 4th ed. Los 32. Muthen ´ and Muthen; ´ 2006. Angeles, CA: Muthen 33. Bollen KA. Modeling strategies: in search of the holy grail. Struct Equ Modeling. 2000;7(1):74-81. 34. Gerbing DW, Anderson JC. On the meaning of withinfactor correlated measurement errors. J Consum Res. 1984;11: 572-580. ´ 35. Muthen B. A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika. 1984;49:115-132. ´ B, Kaplan D. A comparison of some methodologies for 36. Muthen the factor analysis of non-normal Likert variables. Br J Math Stat Psychol. 1985;38:171-189. ´ 37. Muthen, B, du Toit SHC, Spisic D. Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. Technical Report. 1997. Available at: http://www.statmo del.com/bmuthen/articles/Article_075.pdf. Accessed November 29, 2014. 38. Nussbeck FW, Eid M, Lischetzke T. Analysing multitraitmultimethod data with structural equation models for ordinal variables applying the WLSMV estimator: what sample size is needed for valid results? Br J Math Stat Psychol. 2006;59:195-213. 39. Shelvin M, Miles JNV, Davies MNO, Waker S. Coefficient alpha: a useful indicator of reliability? Pers Individ Dif . 2000;28:229239. 40. Rubin DB. Inference and missing data. Biometrika. 1976;63:581592. 41. Grimm LG, Yarnold PR, eds. Reading and Understanding More Multivariate Statistics. Washington, DC: American Psychological Association; 2000. 42. Tucker LR, Lewis C. The reliability coefficient for maximum likelihood factor analysis. Psychometrika. 1973;38:1-10. 43. Steiger JH. A note on multiple sample extensions of the RMSEA fit index. Struct Equ Modeling. 1998;5:411-419. 44. Steiger JH, Lind JC. Statistically based tests for the number of common factors. Paper presented at the annual meeting of the Psychometric Society. Iowa City, IA; May 1980. 45. Byrne BM. Structural Equation Modeling With LISREL, PRELIS, and SIMPLIS: Basic Concepts, Applications, and Programming. Mahwah, NJ: Erlbaum; 1998.

March 2015, Vol. 85, No. 3 •

© 2015, American School Health Association • 187

46. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6:1-55. 47. Nunnally J, Bernstein I. Psychometric Theory. 3rd ed. New York, NY: McGraw Hill; 1994. 48. Lee VE, Smith JB. Social support and achievement for young adolescents in Chicago: the role of school academic press. Am Educ Res J. 1999;36:907-945. 49. Skinner EA, Belmont MJ. Motivation in the classroom: reciprocal effects of teacher behavior and student engagement across school year. J Educ Psychol. 1993;85(4):571-581. 50. Brookmeyer KA, Fanti KA, Henrich CC. Schools, parents, and youth violence: a multilevel, ecological analysis. J Clin Child Adolesc Psychol. 2006;35(4):504-514. 51. Koth CW, Bradshaw CP, Leaf PJ. A multilevel study of predictors of student perceptions of school climate: the effect of classroomlevel factors. J Educ Psychol. 2008;100(1):96. 52. Loukas A, Suzuki R, Horton KD. Examining school connectedness as a mediator of school climate effects. J Res Adolesc. 2006;16(3):491-502.

188 •

Journal of School Health



March 2015, Vol. 85, No. 3 •

53. Borowsky IW, Ireland M, Resnick MD. Violence risk and protective factors among youth held back in school. Ambul Pediatr. 2002;2:475-484. 54. Shochet IM. School connectedness is an underemphasized parameter in adolescent mental health: results of a community prediction study. J Clin Child Adolesc Psychol. 2006;35(2):170179. 55. Catsambis S. Expanding knowledge of parental involvement in children’s secondary education: connections with high school seniors’ academic success. Soc Psychol Educ. 2001;5:149-177. 56. Chung-Do J, Filibeck K, Goebert D, et al. Understanding students’ perceptions of a high school course designed to enhance school connectedness. J Sch Health. 2013;83:478-484. 57. Metz ALR. Why conduct a program evaluation? Five reasons why evaluation can help an out-of-school time program. Research to Results Brief, 31. 2007. Available at: https://cyfernet search.org/sites/default/files/Child_Trends-2007_10_01_RB_ WhyProgEval.pdf. Accessed November 21, 2011.

© 2015, American School Health Association

Copyright of Journal of School Health is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.

Developing a comprehensive school connectedness scale for program evaluation.

Evidence show that school connectedness is important to youth wellness. However, considerable inconsistency in the concepts and measures of school con...
203KB Sizes 2 Downloads 12 Views