Schooi Psychology Quarterly 2013. Vol. 28. No. 4. 273-276
© 2013 American Psychological Association 1045-3830/13/$12.00 DOI: 10.1037/spq00O0O47
Assessment of General Education Teachers' Tier 1 Classroom Practices: Contemporary Science, Practice, and Policy Linda A. Reddy
Gregory A. Fabiano
University at Buffalo
Shane R. Jimerson University of California at Santa Barbara Progress monitoring is a type of formative assessment. Most work on progress monitoring in elementary school settings has been focused on students. However, teachers also can benefit from frequent evaluations. Research addressing teacher progress monitoring is critically important given the recent national focus on teacher evaluation and effectiveness. This special topic section of School Psychology Quarterly is the first to showcase the current research on measuring Tier 1 instructional and behavioral management practices used by prekindergarten and elementary school teachers in general education settings. The three studies included in the special section describe the development and validation efforts of several teacher observational and self-report measures of instruction and/or behavioral management. These studies provide evidence for the utility of such assessments for documenting the use of classroom practices, and these assessment results may be leveraged in innovative coaching models to promote best practice. These articles also offer insight and ideas for the next generation of teacher practice assessment for the field. Finally, the special topic is capped by a commentary synthesizing the current work and offers "big ideas" for future measurement development, policy, and professional development initiatives.
There is increased attention on teacher accountability and the use of evidence-based instructional and behaviorally supportive interventions in schools (e.g.. Bales, 2006; Ysseldyke & Bums, 2009). With the recent Individuals with Disabilities Education Act (IDEA) reauthorization and its emphasis on a Response to Intervention (Rtl) franaework, school psycholo-
Linda A. Reddy, Department of Graduate School of Applied and Professional Psychology, Rutgers University; Gregory A. Fabiano, Department of Counseling, School, and Educational Psychology, University at Buffalo; Shane R. Jimerson, Department of Counseling, Clinical, and School Psychology, University of California at Santa Barbara. The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A080337 to Rutgers University. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education. Correspondence concerning this article should be addressed to Shane R. Jimerson, University of California at Santa Barbara, GGSE-2113, Santa Barbara, CA 93106. E-mail: [email protected]
gists and school administrators are increasingly pressed to document, monitor, and evaluate the interventions and approaches used by general educators (Jimerson, Burns, & VanDerHeyden, 2(X)7). Underscoring this emphasis on the practice of the general educator is the current Secretary of Education stating, "The quality of the teacher in the classroom is the single biggest in-school influence on student learning" (Duncan, Gurria, & van Leeuwen, 2011). Although most would agree that the use of best classroom practices in general education classrooms have always been the gold standard aspiration for educators, usage likely varies across classrooms and contexts, and these recent changes in legislation have only increased the importance of efficiently and accurately documenting what teachers do in classrooms. For decades, research has underscored the effect of teacher knowledge, skills, and strategies on student learning and behavior (e.g., Brophy & Good, 1986; Hall, Panyan, Rabon, & Broden, 1968; Heward, 2003; O'Leary & O'Leary, 1972; Stichter et al., 2009; Sugai & Homer, 2007; Sutherland, Wehby, & Yoder,
REDDY, FABIANO, AND JIMERSON
2002). Teachers' use of specific instructional and classroom behavior management strategies is strongly related to the academic and behavioral success of school-age children (e.g.. Gable, Hester, Rock, & Hughes, 2009; Leflot, van Lier, Onghena, & Colpin, 2010; Wang, Haertel, & Walhherg, 1993). Despite this research evidence, little attention has been devoted to developing and validating methods to practically monitor teachers' use of such strategies for improving professional functioning (Baker et al., 2010; Reddy, Fabiano, Barharasch, & Dudek, 2012; Reddy, Fabiano, & Dudek, 2013; Reddy, Newman, DeThomas, & Chun, 2009). Most teacher measures have been developed for specific interventions focused on single-domain strategies or single strategies rather than daily classroom practices, without consumer input, and for individuals external to the school (e.g., researchers, students on internship or practicum). This is problematic because teachers' strategies work together in concert rather as solo events. Only a comprehensive measure can provide accurate and informative information on teachers' current strategy use as well as highlight areas for growth. Unfortunately, despite this need, the current state of educational science and practice offers few feasible alternatives for school districts interested in assessing and promoting teacher instructional and behavior management practices. Baker et al. (2010) stated in their assessment of the state of the field in measuring teacherrelated outcomes, ".. . American public schools generally do a poor job of systematically developing and evaluating teachers" (p. 1). Assessment approaches that focus solely on student test scores have been criticized as unreliable and susceptible to third-variable influences often outside of the control of the teacher (e.g., student attendance, school-level effects; Corcoran, 2010). The assessment of teacher practices requires an appreciation for the complexity of teaching, the learning process, and the reciprocal and recursive fashion in which educators' use strategies (e.g., provision of academic response opportunities, metacognitive questioning, cooperative learning, and academic corrective feedback) to orchestrate rich and productive social learning environments. This dynamic process also requires tools that are designed for school personnel that are practical (brief and user friendly) for routine educational practice if
they are going to be effective for monitoring progress over time. It is also important to consider how the purpose of assessment informs measurement. Mash and Hunsley (2005) cogently noted that measures must be clearly aligned with the purpose of the assessment. In the field of education, there is currently a disconnect between some measures of teacher outcome and their purpose. Although summative assessments of student achievement are useful for documenting what children have learned, they are not a measure of what teachers do or should do. It may be more effective in the long run, and helpful to educators, to develop and focus assessments on proximal goals under the control of the educator rather than equifinal distal outcomes. Thus, one purpose of this special topic section of School Psychology Quarterly is to present approaches that may have merit for supporting teachers and other school personnel in improving classroombased instructional and/or behavior management strategies and in turn investigate how these measures relate to more distal outcomes such as academic achievement. Scope of the Current Special Topic Section The articles included in the special topic section of this issue of School Psychology Quarterly represent the current science for teacher assessment of Tier 1 classroom practices. Each article includes practical tools that are theoretically grounded, psychometrically sound, and of practical utility for identifying and monitoring teacher practices. The complexity of measuring teacher practices (behaviors) and changing teacher practices is clearly articulated in the articles in this special topic. As noted, many of the articles were supported by federal and foundation grants for supporting this very purpose. For example, the article by Crawford, Zucker, Williams, Bhavsar, and Landry (2013) presents the empirical foundation of the Classroom Observational Tool (COT) and how the COT can be used in a collaborative coaching model to promote effective instructional practices in over 3,900 early childhood teachers. These exciting findings highlight that coaches can be successfully trained to use an evidence-based observation tool to systematically and practically set goals with teachers. This important work is being used for further
MEASURING TEACHER OUTCOMES
revising the COT for future professional development studies. The article by Reddy, Fabiano, Dudek, and Hsu (2013a) presents the development and investigation of the construct validity of a new classroom observation assessment (Classroom Strategies Scale; CSS) that measures classroom instructional and behavioral management practices. A carefully prescribed test development process is outlined. The article describes the theoretical basis and initial reliability and validity estimates of the CSS. This body of work is being used to further revise the CSS for future large-scale validation, teacher evaluation, and professional development efforts. The article by Reddy, Fabiano, Dudek, and Hsu (2013b) presents the initial predictive validity of the CSS for student statewide testing in mathematics and English language arts. An important consideration for any assessment of teacher professional practice in the current climate of high-stakes testing and teacher accountability emphasizes the relationship between classroom practice and student outcomes. In an initial investigation, the teachers' academic instruction and behavior management strategy use were used to predict classwide outcomes on statewide assessments. Not surprisingly, given the robust support for effectiveness of these strategies in the larger literature, some promising outcomes were obtained in which teachers using strategies more consistent with recommendations from the larger field had students who performed better on assessments. The implications for incorporating these measures into professional learning communities to promote improved student achievement are discussed in light of these findings. Finally, the commentary provided hy Conner (2013) further contributes to the special topic section by discussing "big ideas" and offers recommendations for researchers, school personnel, and policy-makers on teacher assessment approaches for evaluating educator effectiveness and informing professional development in schools. Conner emphasizes that the two observation systems presented in this special topic section provide important empirical knowledge regarding assessment systems that can inform and advance policy that could (a) make a difference in what is defined as high-quality and effective teaching; (b) clarify what it looks like in the classroom; and (c)
ultimately he used to promote instructional practices such that all children can experience effective instruction, academic success, and the lifelong accomplishment that follows. In summary, the primary purpose of this special topic section is to present the current science and practice of teacher assessment of general education teachers' Tier 1 practices. Recent national attention and policies on teacher evaluation and effectiveness increase the relevance and urgency of this special topic. Statewide mandates are requiring schools to adopt teacher practice assessments for evaluating educator effectiveness, which in tum are being used to make human capital decisions (i.e., retain, promote, or dismiss), which are decisions no one should take lightly. It is hoped that this special topic helps to refocus the field toward emphasizing eff^ective assessments related to proximal (i.e., teacher competencies) and distal (i.e., students' social and academic behaviors) outcomes that are meaningful and supportive in contrast to the punitive manner in which some assessments are being used within policy and schools at the present time. It is our hope that this special topic section of School Psychology Quarterly will stimulate scholarly debate, future measurement development, and promote the assessment and implementation of classroom practices that improve educator capacities that are ultimately linked to student growth.
References Baker, E. L., Barton, P. E., Darling-Hammond, L., Haertel, H., Ladd, H. F., Linn, R., . . . Shepard, L. A. (2010). Problems with the use of student test scores to evaluate teachers (Briefing Paper #278). Washington, DC: Economic Policy Institute. Bales, B. L. (2006). Teacher education policies in the United States: The accountability shift since 1980. Teaching and Teacher Education, 22, 395-407. doi:10.1016/j.tate.2005.11.009 Brophy, J. E., & Good, T. (1986). Teacher behavior and student achievement. In M. C. Wittrock (Ed.), Handbook of research in teaching (3rd ed., pp. 328-375). New York, NY: Macmillan. Conner, C. M. (2013). Commentary on two classroom observation systems: Moving toward a shared understanding of effective teaching. School Psychology Quarterly, 28, 342-346. Corcoran, S. P. (2010). Can teachers be evaluated by their students' test scores? Should they be? The use of value-added measures of teacher effective-
REDDY, FABIANO, AND JIMERSON
ness in policy and practice. Providence, RI; Annenberg Institute for School Reform at Brown University. Crawford, A. D., Zucker, T. A., Williams, J. M., Bhavsar, V., & Landry, S. H. (2013). Initial validation of the pre-kindergarten Classroom Observation Tool and goal setting system for data-based coaching. School Psychology Quarterly, 28, 277300. Duncan, A., Gurda, A., & van Leeuwen, F. (2011). Uncommon wisdom on teaching. Retrieved July 22, 2011 from http://www.huffingtonpost.com/ arne-duncan/uncommon-wisdom-on-teachi^836541.html Gable, R. A., Hester, P. H., Rock, M. L., & Hughes, K. G. (2009). Back to basics: Rules, praise, ignoring and reprimands revisited. Intervention in School and Clinic, 44, 195-205. doi: 10.1177/ 1053451208328831 Hall, R. V., Panyan, M., Rabon, D., & Broden, M. (1968). Instructing beginning teachers in reinforcement procedures which improve classroom control. Journal of Applied Behavior Analysis, 1, 315-322. doi:10.1901/jaba.l968.1-315 Heward, W. L. (2003). Ten faulty notions about teaching and leaming that hinder the effectiveness of special education. The Journal of Special Education, 36, 186-205. doi: 10.1177/002246690303600401 Jimerson, S. R., Bums, M. K., & VanDerHeyden, A. M. (Eds.) (2007). Handbook of Response to Intervention: The science and practice of assessment and intervention. New York, NY: Springer Science, doi: 10.1007/978-0-387-49053-3 Leflot, G., van Lier, P. A. C, Onghena, P., & Colpin, H. (2010). The role of teacher behavior management in the development of disruptive behaviors: An intervention study with the good behavior game. Journal of Abnormal Child Psychology, 38, 869 - 882. doi : 10.1007/s 10802-010-941 f -4 Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of child and adolescent disorders: Issues and challenges. Journal of Clinical Child and Adolescent Psychology, 34, 362-379. doi:10.1207/ sl5374424jccp3403_l O'Leary, K. D., & O'Leary, S. G. (1972). Classroom management: The successful use of behavior modification. New York, NY: Pergamon Press. Reddy, L., Eabiano, G., Barbarasch, B., & Dudek, C. (2012). Behavior management of students with attention-deficit/hyperactivity disorders using teacher and student progress monitoring. In L. M. Crothers & J. B. Kolbert (Eds.), Understanding and managing behaviors of children with psycho-
logical disorders: A reference for classroom teachers (pp. 17-47). New York, NY: Continuum Intemational. Reddy, L., Eabiano, G., & Dudek, C. (2013). Concurrent validity of the Classroom Strategies Scale-Observer Eorm. Journal of Psycheducational Assessment, 31, 258-270. doi:10.1177/ 0734282912462829 Reddy, L. A., Eabiano, G. Dudek, C , & Hsu, L. (2013a). Development and construct validity of the Classroom Strategies Scale-Observer Eorm. School Psychology Quarterly, 28, 317-341. Reddy, L. A., Eabiano, G., Dudek, C , & Hsu, L. (2013b). Predictive validity of the Classroom Strategies Scale-Observer Eorm on statewide testing scores: An initial investigation. School Psychology Quarterly, 28, 301-316. Reddy, L. A., Newman, E., DeThomas, C , & Chun, V. (2009). Effectiveness of school-based prevention and intervention programs for children and adolescent with emotional disturbance: A metaanalysis. Journal of School Psychology, 47, 77-99. doi:10.1016/j.jsp.2008.11.001 Stichter, J. P., Lewis, T. J., Whittaker, T. A., Richter, M., Johnson, N. W., & Trussell, J. R. (2009). Assessing teacher use of opportunities to respond and effective classroom management strategies: Comparisons among high-and low-risk elementary school. Journal of Positive Behavior Interventions, ¡I, 68-81. Sugai, G., & Homer, R. H. (2007). Is school-wide Positive Behavioral Support an evidence-based practice? Retrieved October 24, 2007 from http:// pbis.org/files/101007evidencebase4pbs.pdf Sutherland, K. S., Wehby, J. H., & Yoder, P. J. (2002). Examination of the relationship between teacher praise and opportunities for students with EBD to respond to academic requests. Journal of Emotional and Behavioral Disorders, 10, 5-13. doi:10.1177/106342660201000102 Wang, M. C , Haertel, G. D., & Walhberg, H. J. (1993). Toward a knowledge base for school leaming. Review of Educational Research, 63, 249294. doi:10.3102/00346543063003249 Ysseldyke, J., & Bums, M. (2009). Eunctional assessment of instructional environments for the purpose of making data-driven instructional decisions. In T. Gutkin & C. Reynolds (Eds.), The handbook of school psychology (4th ed., pp. 410-433). Hoboken, NJ: Wiley. Received October 18, 2013 Accepted October 18, 2013 •
Copyright of School Psychology Quarterly is the property of American Psychological Association and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.