American Journal of Community Psychology, VoL 6, No. 3, 1978
Referral to a School Mental Health Project: A Screening Note 1 Raymond P. Lorion 2 Temple University Emery L. Cowen University of Rochester
The screening efficiency o f two measures o f school-adfustment is considered by assessing the proportion o f children referred to a school mental health program whose scores fall at or above the 85th centile on one or both measures. The study's findings demonstrate that 60-90% o f referred children were identifiable by a screening cutoff score procedure. The implications for using a cutoff score approach in selecting children for the intervention are discussed. The present study evaluates the potential screening efficiency of two referral measures used in a school-based mental health program. Specifically, the authors are assessing the accuracy of using a normatively based criterion score procedure to identify target children. To date, the identification of children has relied entirely upon the sensitivity of classroom teachers. Although this system may be appropriate in settings in which the teaching staff have considerable knowledge of and experience with a program, it must be objectified if the identification and intervention strategies are to be generalized to a range of settings and resource situations. Additionally, need for a careful evaluation of the referral process extends beyond the specific program under study. In the past decade, numerous secondary preventive programs have been designed which share the goals of identifying incipient dysfunction and of miitimizing the negative impact of disorder through effective early remediation. Both for conceptual and pragmatic reasons, 1The present study is based on work supported by giant MH-11820-05 from the NIMH Experimental and Special Training Branch for which the authors express sincere appreciation. 2All correspondence should be sent to Dr. Raymond P. Lotion, Department of Psychology, Temple University, Philadelphia, Pennsylvania 19122. 247 0091-0562/78/0600-0247505.00/0
© 1978 Plenum Publishing Corporation
248
Lorion and Cowen
many of these programs have focused on children and have been school-based (Zax & Specter, 1974). Few, however, have included systematic evaluation of either the overall program or its specific components (Cowen, 1973). Community professionals need such information if they are to weigh, objectively, the relative merits of various preventive approaches in meeting the needs of target populations.
METHOD
Setting T h e present study was conducted as part of the ongoing evaluation of the Primary Mental Health Project (PMHP), a school based program for the early detection and treatment of school maladjustment (Cowen, Trost, Lotion, Dorr, Izzo, & Isaccson, 1975). Teachers refer children whom they perceive as unable to adapt to the socioeducational demands of the classroom to the program by completing one or both referral measures described below. Referred children are seen regularly throughout the academic year by professionally supervised, nonprofessional child-aides. Findings to date suggest that PMHP offers a viable (Cowen et al., 1975; Lorion, Caldwell, & Cowen, 1975a) and efficient (Cowen, Lorion, Kraus, & Dorr, 1974) alternative to the delivery of school mental health services. For that reason, PMHP has become increasingly involved in program expansion and development on a local and national basis (Cowen et al., 1975),
Subjects In mid-October, 1973, first-grade teachers (N = 13) in five PMHP schools located in Rochester and its surrounding suburbs completed AML forms (cf. below) on all children in their classrooms (N = 268). Fifty-five of these children (39 males, 16 females;29 from the city, 26 from the county) were later referred to PMHP for school maladjustment problems. At the time of referral (midNovember) teachers also completed a Classroom Adjustment Rating Scale (cf. below) for each child. Due to incomplete or incorrectly completed forms, the latter data were available for only 50 of the 55 referred first graders which is the sample for the present study.
Measures AML is an l 1-item quick-screening device to identify early school maladaptation (Cowen, Dorr, Clarfield, Kreling, McWilliams,Pohrachi, Pratt, Terrell,
Referral Screening
249
& Wilson, 1973). AML items describe specific classroom problems which are rated by the teacher along a 5-point frequency continuum ranging from "never" (1) to "most or all of the time" (5). Factor analyses of the AML reveals a stable three-factor orthogonal solution: acting-out (5 items), moody-withdrawn (5 items), and learning (1 item) dimensions. An overall index of school maladjustment is computed as a sum of the 11 items. Higher subscale and total scores reflect more serious maladjustment. Psychometric properties are reported elsewhere as are the normative distributions of its subscale and total scale scores (Cowen et al., 1973). Classroom Adjustment Rating Scale is a modified version of the Teacher Referral Form (Clarfield, 1974) that includes 41 behaviorally oriented items describing school adjustment problems (Lorion, Cowen, & Caldwell, 1975b). In referring a child to PMHP, teachers rate "the extent to which the behavior interferes with the child's ability to profit from his school experience" along a 5point scale of problem severity ranging from "not a problem" (1) to "very serious problem" (5). Factor analysis of this scale also indicates a stable threefactor orthogonal solution: acting-out (10 items), shy-anxious (12 items), and learning (14 items). An overall index of school-adjustment is computed as a sum of the 41 items scores. Higher subscale and total scores reflect more serious adjustment problems. Normative and parametric characteristics of subscale and total scores are reported elsewhere (Lorion, Cowen, & Caldwell, 1975b).
Procedure Prior research (Cowen et al., 1974) demonstrated that PMttP saw about 10-15% of the primary graders in its various schools. Therefore, the 85th centile was selected as a reasonable screening cutoff point for the two referral measures. The actual AML and Classroom Adjustment Rating School subscale and total scores of the 50 referred students were compared with these arbitrary cutoff screening scores to determine the relative proportion falling above and below those points.
RESULTS
Table I summarizes the proportions of children falling above or below the 85th centile score on one or both PMHP measures. With respect to the AML, 31/50 (62%) of those ultimately referred to PMHP fall at or above the defmed cutoff score on one or more AML subscore. With respect to the Classroom Adjustment Rating Scale, 42/50 (84%) fall at or above the 85th centile on one or more subscore measures. Finally, 45/50 (90%) of the children referred to PMHP fall at or above this level on at least one subscore of the two measures.
250
Lotion and Cowen
Table I. Proportion of Referred Children Identified At or Above 85th Centile on PMHP Referral Measures Classroom adjustment rating scale AML Above 85th centile Below 85th centile Total
Above 85th centile
Below85th centile
Total
28 (56%)
3 (6%)
31 (62%)
14 (28%) 42 (84%)
5 (10%) 8 (16%)
19 (38%) 50 (100%)
DISCUSSION
The main purpose of this study was to assess the screening accuracy of available measures of school-adjustment. The high degree of overlap between actual program referral and test scores at, or above, the 85th centile suggests that the two referral measures currently used by PMHP efficiently identify children who need service. Notwithstanding the AML's brevity (11 items) and the limited classroom observation period (4-5 weeks) that preceeded teacher completion of this form, 60% of the children later referred to PMHP had AML scores above the 85th centile. The greater length (41 items) of the Classroom Adjustment Rating Scale and teacher sensitivity to the service needs of children at the time of referral combined to enable that measure to identify successfully 84% of the referred children. Taken together, the screening accuracy of the two measures was 90%, a promising rate indeed. The authors do not suggest that cutoff scores on one or both measures should be the sole criterion for referral. Other factors, such as the teacher's and/ or parents' level of concern for a child, can and must be considered. Nevertheless, the use of either (or both) instruments in a screening capacity can efficiently identify children whose adjustment status, classroom adaptation, and appropriateness for intervention should be reviewed. Specific cutoff scores should be selected so as to mirror a program's service potential. Thus, if 10% of the children can be served a 90th centile may be used; in other situations the 85th, 80th, or even 60th centile might be appropriate. Our experience, for example, suggests that suburban children tend to have fewer serious problems than urban children, since relatively more suburban children have incipient or minimal difficulties. Thus, the preventive focus of a screening process can be modified flexibly to serve the needs of a particular target population. An additional prospective benefit of the screening approach proposed is that it reduces somewhat the subjectiveness in decision-making which often characterizes assignment to preventatively oriented programs. The allocation of scarce services is always a difficult process.
Referral Screening
251
Hopefully, the more systematized and objective a referral system becomes, the more likely it will be that children in need will receive appropriate remediation.
REFERENCES ClarfMd, S. P. The development of a teacher referral form for identifyingearly school maladaptation. American Journal of Community Psychology, 1974, 2, 199-210. Cowen, E. L. Social and community interventions. Annual Review of Psychology, 1973, 24, 423-472. Cowen, E. L., Doff, D., Clarfield, S. P., Kxeling, B., McWilliams,S. A., Pokraki, F., Pratt, D. M., Terrell, D. L., & Wilson, A. B. The AML: A quick screening device for early detection of school maladaptation. American Journal o f Community Psychology, 1973, 1, 12-35. Cowen, E. L., Lorion, R. P., Kraus, R. M., & Doff, D. Geometric expansion of helping services. Journal of School Psychology, 1974,12, 288-295. Cowen, E. L., Trost, M. A., Lotion, R. P., Dorr, D., Izzo, L. D., & Isaccson, R. V. New ways in school mental health: Early detection and prevention of school maladaptation. New York: Behavioral Publication, 1975. Lotion, R. P., Caldwell, R. A., & Cowen, E. L. Effects of a school mental health project: A one-year follow-up. Journal of School Psychology, 1975, 14, 56-63. (a) Lotion, R. P., Cowen, E. L., & CaldweU, R. A. Normative and parametric analyses of school maladjustment. American Journal o f Community Psychology, 1975, 3, 291-301. (b) Zax, M., & Cowen, E. L. Abnormal psychology: Changing conceptions. New York: Holt, Rinehart & Winston, 1972. Zax, M., & Specter, G. An introduction to community psychology. New York: Wiley, 1974.
Referral to a School Mental Health Project: A Screening Note 1 Raymond P. Lorion 2 Temple University Emery L. Cowen University of Rochester
The screening efficiency o f two measures o f school-adfustment is considered by assessing the proportion o f children referred to a school mental health program whose scores fall at or above the 85th centile on one or both measures. The study's findings demonstrate that 60-90% o f referred children were identifiable by a screening cutoff score procedure. The implications for using a cutoff score approach in selecting children for the intervention are discussed. The present study evaluates the potential screening efficiency of two referral measures used in a school-based mental health program. Specifically, the authors are assessing the accuracy of using a normatively based criterion score procedure to identify target children. To date, the identification of children has relied entirely upon the sensitivity of classroom teachers. Although this system may be appropriate in settings in which the teaching staff have considerable knowledge of and experience with a program, it must be objectified if the identification and intervention strategies are to be generalized to a range of settings and resource situations. Additionally, need for a careful evaluation of the referral process extends beyond the specific program under study. In the past decade, numerous secondary preventive programs have been designed which share the goals of identifying incipient dysfunction and of miitimizing the negative impact of disorder through effective early remediation. Both for conceptual and pragmatic reasons, 1The present study is based on work supported by giant MH-11820-05 from the NIMH Experimental and Special Training Branch for which the authors express sincere appreciation. 2All correspondence should be sent to Dr. Raymond P. Lotion, Department of Psychology, Temple University, Philadelphia, Pennsylvania 19122. 247 0091-0562/78/0600-0247505.00/0
© 1978 Plenum Publishing Corporation
248
Lorion and Cowen
many of these programs have focused on children and have been school-based (Zax & Specter, 1974). Few, however, have included systematic evaluation of either the overall program or its specific components (Cowen, 1973). Community professionals need such information if they are to weigh, objectively, the relative merits of various preventive approaches in meeting the needs of target populations.
METHOD
Setting T h e present study was conducted as part of the ongoing evaluation of the Primary Mental Health Project (PMHP), a school based program for the early detection and treatment of school maladjustment (Cowen, Trost, Lotion, Dorr, Izzo, & Isaccson, 1975). Teachers refer children whom they perceive as unable to adapt to the socioeducational demands of the classroom to the program by completing one or both referral measures described below. Referred children are seen regularly throughout the academic year by professionally supervised, nonprofessional child-aides. Findings to date suggest that PMHP offers a viable (Cowen et al., 1975; Lorion, Caldwell, & Cowen, 1975a) and efficient (Cowen, Lorion, Kraus, & Dorr, 1974) alternative to the delivery of school mental health services. For that reason, PMHP has become increasingly involved in program expansion and development on a local and national basis (Cowen et al., 1975),
Subjects In mid-October, 1973, first-grade teachers (N = 13) in five PMHP schools located in Rochester and its surrounding suburbs completed AML forms (cf. below) on all children in their classrooms (N = 268). Fifty-five of these children (39 males, 16 females;29 from the city, 26 from the county) were later referred to PMHP for school maladjustment problems. At the time of referral (midNovember) teachers also completed a Classroom Adjustment Rating Scale (cf. below) for each child. Due to incomplete or incorrectly completed forms, the latter data were available for only 50 of the 55 referred first graders which is the sample for the present study.
Measures AML is an l 1-item quick-screening device to identify early school maladaptation (Cowen, Dorr, Clarfield, Kreling, McWilliams,Pohrachi, Pratt, Terrell,
Referral Screening
249
& Wilson, 1973). AML items describe specific classroom problems which are rated by the teacher along a 5-point frequency continuum ranging from "never" (1) to "most or all of the time" (5). Factor analyses of the AML reveals a stable three-factor orthogonal solution: acting-out (5 items), moody-withdrawn (5 items), and learning (1 item) dimensions. An overall index of school maladjustment is computed as a sum of the 11 items. Higher subscale and total scores reflect more serious maladjustment. Psychometric properties are reported elsewhere as are the normative distributions of its subscale and total scale scores (Cowen et al., 1973). Classroom Adjustment Rating Scale is a modified version of the Teacher Referral Form (Clarfield, 1974) that includes 41 behaviorally oriented items describing school adjustment problems (Lorion, Cowen, & Caldwell, 1975b). In referring a child to PMHP, teachers rate "the extent to which the behavior interferes with the child's ability to profit from his school experience" along a 5point scale of problem severity ranging from "not a problem" (1) to "very serious problem" (5). Factor analysis of this scale also indicates a stable threefactor orthogonal solution: acting-out (10 items), shy-anxious (12 items), and learning (14 items). An overall index of school-adjustment is computed as a sum of the 41 items scores. Higher subscale and total scores reflect more serious adjustment problems. Normative and parametric characteristics of subscale and total scores are reported elsewhere (Lorion, Cowen, & Caldwell, 1975b).
Procedure Prior research (Cowen et al., 1974) demonstrated that PMttP saw about 10-15% of the primary graders in its various schools. Therefore, the 85th centile was selected as a reasonable screening cutoff point for the two referral measures. The actual AML and Classroom Adjustment Rating School subscale and total scores of the 50 referred students were compared with these arbitrary cutoff screening scores to determine the relative proportion falling above and below those points.
RESULTS
Table I summarizes the proportions of children falling above or below the 85th centile score on one or both PMHP measures. With respect to the AML, 31/50 (62%) of those ultimately referred to PMHP fall at or above the defmed cutoff score on one or more AML subscore. With respect to the Classroom Adjustment Rating Scale, 42/50 (84%) fall at or above the 85th centile on one or more subscore measures. Finally, 45/50 (90%) of the children referred to PMHP fall at or above this level on at least one subscore of the two measures.
250
Lotion and Cowen
Table I. Proportion of Referred Children Identified At or Above 85th Centile on PMHP Referral Measures Classroom adjustment rating scale AML Above 85th centile Below 85th centile Total
Above 85th centile
Below85th centile
Total
28 (56%)
3 (6%)
31 (62%)
14 (28%) 42 (84%)
5 (10%) 8 (16%)
19 (38%) 50 (100%)
DISCUSSION
The main purpose of this study was to assess the screening accuracy of available measures of school-adjustment. The high degree of overlap between actual program referral and test scores at, or above, the 85th centile suggests that the two referral measures currently used by PMHP efficiently identify children who need service. Notwithstanding the AML's brevity (11 items) and the limited classroom observation period (4-5 weeks) that preceeded teacher completion of this form, 60% of the children later referred to PMHP had AML scores above the 85th centile. The greater length (41 items) of the Classroom Adjustment Rating Scale and teacher sensitivity to the service needs of children at the time of referral combined to enable that measure to identify successfully 84% of the referred children. Taken together, the screening accuracy of the two measures was 90%, a promising rate indeed. The authors do not suggest that cutoff scores on one or both measures should be the sole criterion for referral. Other factors, such as the teacher's and/ or parents' level of concern for a child, can and must be considered. Nevertheless, the use of either (or both) instruments in a screening capacity can efficiently identify children whose adjustment status, classroom adaptation, and appropriateness for intervention should be reviewed. Specific cutoff scores should be selected so as to mirror a program's service potential. Thus, if 10% of the children can be served a 90th centile may be used; in other situations the 85th, 80th, or even 60th centile might be appropriate. Our experience, for example, suggests that suburban children tend to have fewer serious problems than urban children, since relatively more suburban children have incipient or minimal difficulties. Thus, the preventive focus of a screening process can be modified flexibly to serve the needs of a particular target population. An additional prospective benefit of the screening approach proposed is that it reduces somewhat the subjectiveness in decision-making which often characterizes assignment to preventatively oriented programs. The allocation of scarce services is always a difficult process.
Referral Screening
251
Hopefully, the more systematized and objective a referral system becomes, the more likely it will be that children in need will receive appropriate remediation.
REFERENCES ClarfMd, S. P. The development of a teacher referral form for identifyingearly school maladaptation. American Journal of Community Psychology, 1974, 2, 199-210. Cowen, E. L. Social and community interventions. Annual Review of Psychology, 1973, 24, 423-472. Cowen, E. L., Doff, D., Clarfield, S. P., Kxeling, B., McWilliams,S. A., Pokraki, F., Pratt, D. M., Terrell, D. L., & Wilson, A. B. The AML: A quick screening device for early detection of school maladaptation. American Journal o f Community Psychology, 1973, 1, 12-35. Cowen, E. L., Lorion, R. P., Kraus, R. M., & Doff, D. Geometric expansion of helping services. Journal of School Psychology, 1974,12, 288-295. Cowen, E. L., Trost, M. A., Lotion, R. P., Dorr, D., Izzo, L. D., & Isaccson, R. V. New ways in school mental health: Early detection and prevention of school maladaptation. New York: Behavioral Publication, 1975. Lotion, R. P., Caldwell, R. A., & Cowen, E. L. Effects of a school mental health project: A one-year follow-up. Journal of School Psychology, 1975, 14, 56-63. (a) Lotion, R. P., Cowen, E. L., & CaldweU, R. A. Normative and parametric analyses of school maladjustment. American Journal o f Community Psychology, 1975, 3, 291-301. (b) Zax, M., & Cowen, E. L. Abnormal psychology: Changing conceptions. New York: Holt, Rinehart & Winston, 1972. Zax, M., & Specter, G. An introduction to community psychology. New York: Wiley, 1974.
![[Mental health promotion in school with school coaches: evaluation of a pilot project].](/assets/img/hold.png)
