This article was downloaded by: [New York University] On: 28 May 2015, At: 05:49 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Research Quarterly for Exercise and Sport Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/urqe20

Effects of Improving Teachers' Content Knowledge on Teaching and Student Learning in Physical Education a

b

c

Phillip Ward , Insook Kim , Bomna Ko & Weidong Li a

The Ohio State University

b

Kent State University

a

c

East Carolina University Published online: 24 Dec 2014.

Click for updates To cite this article: Phillip Ward, Insook Kim, Bomna Ko & Weidong Li (2015) Effects of Improving Teachers' Content Knowledge on Teaching and Student Learning in Physical Education, Research Quarterly for Exercise and Sport, 86:2, 130-139, DOI: 10.1080/02701367.2014.987908 To link to this article: http://dx.doi.org/10.1080/02701367.2014.987908

PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions

Research Quarterly for Exercise and Sport, 86, 130–139, 2015 Copyright q SHAPE America ISSN 0270-1367 print/ISSN 2168-3824 online DOI: 10.1080/02701367.2014.987908

Effects of Improving Teachers’ Content Knowledge on Teaching and Student Learning in Physical Education Phillip Ward The Ohio State University

Insook Kim Downloaded by [New York University] at 05:49 28 May 2015

Kent State University

Bomna Ko East Carolina University

Weidong Li The Ohio State University

Purpose: The purpose of this study was to examine the efficacy of a content knowledge (CK) workshop on the enacted pedagogical content knowledge (PCK) of teachers and in turn the effects on student learning. Method: A quasiexperimental design was used to examine 4 questions: (a) How does student learning differ as a function of PCK? (b) How does teachers’ selection of tasks differ in teaching as a function of CK? (c) How does teachers’ representation of tasks differ in teaching as a function of CK? (d) How does teachers’ adaptation of tasks differ in teaching as a function of CK? In examining student learning, 2 statistical analyses were employed using correct trials and incorrect trials as dependent measures, respectively. Analyses of variance were conducted examining the effects of the intervention on students’ percentage of correct or incorrect trials. The analyses consisted of independent variables, including teachers as a block, treatment, class nested within conditions, gender, skill levels, and 2-way interactions among treatment conditions, gender, and skill levels. Results: A statistically significant effect was reported for both analyses. Effect sizes were .63 and .67, indicating a moderate-to-high practical difference between groups in favor of the experimental condition. Large effect size differences (. 2.0) were found for teacher PCK behaviors. There were no interaction effects. Conclusion: Our findings show that the enacted PCK of a teacher can be changed from immature to mature as a function of learning CK and that this change has a significant and meaningful impact on student learning. Keywords: pedagogical content knowledge, professional development, teaching effectiveness

Teaching physical education effectively is extraordinarily complex work, requiring deep understanding such as knowledge of the characteristics of the students being taught, how learning occurs, the content to be taught, differentiated Submitted January 28, 2014; accepted September 3, 2014. Correspondence should be addressed to Phillip Ward, Department of Human Sciences, The Ohio State University, Rm. 256 PAES Building, 305 West 17th Avenue, Columbus, OH 43210-1221. E-mail: ward.116@osu. edu

pedagogy, and curricula (Shulman, 1986). One of the more useful concepts for examining the practice of teaching is pedagogical content knowledge (PCK; Shulman, 1986). PCK is that form of content knowledge (CK) where a teacher transforms content (e.g., the serve in tennis) in ways that help students learn and understand the content (e.g., using a ball hanging from a stick to the right and above the head of the student to show the point of contact in the serve). PCK has evolved from Shulman’s (1986) initial conceptualization consisting of an amalgamation of pedagogy and content, to

EFFECTS OF IMPROVING TEACHERS’ CONTENT KNOWLEDGE

Downloaded by [New York University] at 05:49 28 May 2015

Grossman’s (1990) widely accepted concept of PCK as an amalgam of four teacher knowledge bases: (a) knowledge and beliefs about the purposes and goals for teaching, (b) knowledge of students’ understanding of the subject matter, (c) knowledge of curricula and curricular resources, and (d) knowledge of representations and instructional strategies. Most recently, PCK has been conceptualized in terms of knowledge of content and students, knowledge of content and teaching, and knowledge of curriculum (Ball, Thames, & Phelps, 2008). Regardless of its conceptualization, when PCK is described in research articles, some version of Shulman’s (1987, p. 9) description is often used: The most useful forms of representation of those ideas, the most powerful analogies, illustrations, examples, explanations and demonstrations—in a word, the most useful ways of representing and formulating the subject that makes its comprehensible to others . . . pedagogical content knowledge also includes an understanding of what makes the learning of specific topics easy or difficult, the conceptions and preconceptions that students of different ages and backgrounds bring with them to the learning of those most frequently taught topics and lessons.

However, these conceptualizations and descriptions, which are useful as heuristic devices, are less useful as an operational definition. This has led to a widespread criticism that definitions of PCK are unclear and too broad to be useful to researchers and teachers (Abell, 2008; Ball et al., 2008; Marks, 1990). In addition to this criticism, we argue that an implicit assumption in Shulman’s (1987) work—and in fact, most definitions of PCK—is that PCK leads to improved student learning. This assumption, however, does not explain how an instance of PCK might be effective for some students and not for others. Nor does it explain varying ranges in the quality of instances of PCK used by teachers that have been reported in the physical education literature (e.g., A. Chen & Ennis, 1995; Hastie & Vlaisavljevic, 1999; Jenkins & Veal, 2002). These problems can be resolved using the existing literature. First, PCK can be operationally defined as the instructional representations and tasks enacted, planned, or described by the teacher based on Shulman’s own description of PCK’s purpose: “How the content is organized, represented and adapted to the diverse interests and abilities of the learners and presented for instruction” (Shulman, 1987, p. 8). Thus, the representation of a task (e.g., demonstrations and instructions) and the task to be performed become the focal point of the analysis when one examines enacted PCK. This includes both intertask and intratask adaptations (Ayvazo & Ward, 2011), and the organization of the content in terms of the lesson, the unit, and a school curriculum can be empirically examined in terms of the relationship between representations and tasks.

131

Drawing upon this operational definition, our work on this project has been informed by our conceptualization of PCK: PCK is a focal point, a locus, defined as such as an event in time (and therefore specific contextually) where teachers make decisions in terms of content based on their understandings of a number of knowledge bases (e.g., pedagogy, learning, motor development, students, contexts, and curriculum).

This conceptualization recognizes and emphasizes that PCK represents the teacher’s efforts to match content to student learning. This matching or transformation is informed by the different knowledge bases. A second solution to the problems posed is Rovegno’s (1992) conceptualization of PCK as knowledge that varies in its maturity. Recognizing that PCK varies in its maturity explains the differential effectiveness of PCK relative to student learning. In short, all teaching representations and the tasks the students perform become the primary units to analyze when determining maturity of PCK. Thus one measure of PCK is the enacted teaching representations and tasks that can be seen to vary in their maturity. The literature has consistently shown that PCK varies in its maturity. Researchers in describing PCK have used terms such as “developed” (A. Chen & Ennis, 1995, p. 391), “weak and strong” (Hastie & Vlaisavljevic, 1999, p. 30), and “mature, immature, and inadequate” (Ayvazo & Ward, 2011, pp. 675 – 676; W. Chen & Rovegno, 2004, p. 194). The conclusion is that PCK can be considered as a class of behaviors that develops on a continuum, from immature to mature forms (Ayvazo & Ward, 2011; Rovegno, 1992). If one accepts that PCK varies in maturity, the relevant question becomes: What is the cause of the variation? If PCK is informed by a variety of knowledge bases, then varying depths of understanding among those knowledge bases presumably influences PCK. This assumption notwithstanding, one of the most universal conclusions across subject areas is that PCK is strongly influenced by CK (Ayvazo & Ward, 2011; Ball et al., 2008; W. Chen & Rovegno, 2004; Grossman, 1990; Jenkins & Veal, 2002; McCaughtry & Rovegno, 2003; Schempp, Manross, Tan, & Fincher, 1998). However, this relationship between CK and PCK is hypothesized; to date, there are no direct casual links. One reason for this has been the problem of the conceptual clarity of CK. Two recent conceptualizations of CK clarify the relationship between CK and PCK. Ball et al.’s (2008) distinction between common content knowledge (CCK) and specialized content knowledge (SCK) and Ward’s (2009) conceptualization of CK for physical activity provide operational definitions that in turn allow CK to be linked to PCK. Ball et al. (2008) have classified CK into two broad domains: (a) CCK, which refers to the knowledge and skills one needs to perform a task such as using the correct

Downloaded by [New York University] at 05:49 28 May 2015

132

P. WARD ET AL.

technique to perform the layup in basketball; and (b) SCK, which refers to knowledge and skills that represent how to teach the layup in basketball. Ward (2009) described CCK as knowledge of the rules and etiquette and knowledge of techniques and tactics, and described SCK as knowledge of student errors and knowledge of instructional tasks and representations. In short, SCK is that form of CK that represents a teacher’s understanding of the tasks that can be used to teach CCK. To elaborate further, consider a textbook describing the sequence of tasks for teaching the front crawl in swimming. This represents CCK (e.g., the technique of the stroke) and SCK (e.g., instructional tasks). The description is purely content. A teacher reading that content and preparing to teach beginners would select very different tasks than if she was preparing to teach students who were more advanced. In both cases, the teacher would draw on knowledge other than content to make decisions about what content to teach such as her understanding of students’ abilities and learning characteristics, the space and equipment in the pool, and the pedagogy to be used (e.g., whole group, self-directed, or partner instruction). Bringing these different understandings together to select what content to include and how it is to be taught to specific students in particular contexts is PCK. To date, most conclusions about CK and PCK have been hypothetical. The lack of causal evidence studies demonstrating how CK impacts PCK leaves critical gaps in our theoretical knowledge, and this hinders our ability to guide the practice of teacher education in both preservice and continuing teacher education settings. The purpose of this study was to examine the relationship between CK and PCK. We hypothesized that increasing SCK, in particular, would result in improved teaching performance and subsequent improvement of student performance. Our research questions were: (a) How does student achievement differ as a function of PCK? (b) How do teachers’ selections of tasks differ as a function of CK? (c) How do teachers’ representations of tasks differ as a function of CK? (d) How do teachers’ adaptations of tasks differ as a function of CK?

METHOD Research Design A quasiexperimental design was used to examine teachers’ enacted PCK and student learning before and after a badminton CK workshop. Each of the four teachers, as a block in our design, taught four classes that were randomly assigned to either a control or an experimental condition. The study was organized in three phases. In the first and third phases, we observed student performances live and observed teacher behaviors from digital videos during a 6-day badminton unit with two classes per teacher. In the second phase, teachers participated in a CK workshop in

each school site. In both the first and third phases, the teachers were asked to organize their lessons similarly with: (a) a 7-min to 8-min warm-up (i.e., any kind of activities for warming up), (b) a 15-min to 20-min practice (i.e., activities for developing specific skills and tactics either in the practice or game situation), and (c) a 10-min to 15-min game play (i.e., activities for applying the learned skills or tactics into a modified or real game). Participants The study protocol received approval from each site’s university institutional review board. Consent was obtained from the district, teachers, and parents. Assent was obtained from students. Teachers Four middle school physical education teachers were purposefully selected as participants for this study from two public suburban middle schools located in North Carolina and Ohio according to the following criteria: (a) They did not consider badminton as an expert content area; (b) they were able to teach two 6-day badminton lessons to four classes (i.e., six lessons for two comparison classes before the CK workshop and another six lessons for two experimental classes after the CK workshop); and (c) they were willing to participate in the study. All four teachers were male and Caucasian. The age of the teachers ranged from 34 years to 47 years old, and teaching experience ranged from 4 to 20 years. Students We used the teachers’ intact classes, and all students were considered potential participants with the exception of students whom the teachers indicated were frequently absent or misbehaved. These students were excluded to minimize the loss of data. The average class size was 21 (range ¼ 18 –24). In each class, 6 students were randomly selected from stratified skill-level groups (i.e., 2 highskilled, 2 middle-skilled, and 2 low-skilled students—1 of each sex) grouped by each teacher. A total of 96 students (i.e., 4 teachers £ 4 classes £ 6 students) who were in sixth grade (n ¼ 24), seventh grade (n ¼ 48), and eighth grade (n ¼ 24) participated in this study. Students were equally distributed in terms of gender (n ¼ 48 male and n ¼ 48 female). In terms of ethnicity, the students were 72% (n ¼ 69) White, 27% Black (n ¼ 26), and 1% Hispanic (n ¼ 1). Independent Variables The independent variable was a package intervention consisting of two components: (a) the badminton CK

EFFECTS OF IMPROVING TEACHERS’ CONTENT KNOWLEDGE

workshop, and (b) teaching feedback provided to the teachers following lessons on a daily basis.

Downloaded by [New York University] at 05:49 28 May 2015

The Content Knowledge Workshop The content for the workshop was presented to the teachers in a “knowledge packet” developed using the books Play Practice (Launder, 2001) and Badminton Steps to Success (Grice, 2008). To further ensure the validity of the content in the knowledge packet, we requested two badminton experts who had been long-term teachers and players of badminton to thoroughly check whether: (a) the sequence of the tasks was appropriate, and (b) the content was developmentally appropriate. An expert in play practice was used to assess the content to ensure it was consistent with the principles of play practice. The knowledge packet focused on six badminton skills: (a) serving, (b) overhead strokes, (c) underhand strokes, (d) smash shots, (e) drop shots, and (f) double strategies, which were included to facilitate teachers’ badminton CK. The content of the knowledge packet included a set of task progressions for teaching the badminton skills (SCK), the list of critical elements of each skill and tactic (CCK), the list of common errors and error corrections (SCK), and a recommendation on how to sequence tasks for a 6-day badminton unit (SCK). Prior to the start of the workshop, the knowledge packet was provided to the teachers and they were requested to review it. The focus of the workshop was to develop the teachers’ CCK and predominantly SCK of badminton. The duration of the workshop was 4 hr, and it conducted across 1 or 2 days depending on the preferences of the teachers. The workshop consisted of three components: (a) an overview and introduction, (b) watching the workshop video, and (c) evaluation, which occurred throughout the workshop. Teachers were provided with an overview of the workshop including (a) the purpose of the workshop, (b) expectations, and (c) a discussion of the principles of play practice (Launder, 2001). Next, the teachers watched the workshop video that presented: (a) the objective of the instructional task, (b) examples of what were appropriate tasks using the principles taught in the introduction (SCK), (c) specific and sequenced task progressions for teaching the badminton unit (SCK), (d) critical elements of each skill and tactic (CCK), (e) examples of verbal and visual representations of the tasks (SCK), and (f) task adaptations of the content sequence for students who had lower abilities and skill levels (SCK). After these elements (a – f) were presented for a task, the video was paused and teachers were asked to demonstrate correct techniques and answer questions regarding skill technique (CCK), error detection and corrections (SCK), task representations (SCK), task progressions (SCK), or task adaptations for student learning (SCK). We evaluated teacher knowledge of each element (a – f) after each skill was presented in the workshop. In all, teachers were asked 64 questions, and we used an overall

133

criterion of 96% correct to determine their understanding of the content. When the teachers did not meet the criterion on the first trial, we reviewed the workshop content again with them. Then the teachers reanswered the questions that they missed on the first trial. Examples of questions that were asked were: “Could you please describe five critical elements of a forehand long serve?” (technique). “Can you describe the common errors made by students when performing backhand overhead stroke?” (error detection). “Can you describe the ways in which these errors might be corrected?” (correction). The answers were judged relative to the CK domains (Ward, 2009) that were presented in the workshop. Teaching Feedback After each lesson with the experimental classes (i.e., postworkshop), the teachers received verbal feedback about their teaching in terms of their selection, representation, and adaptation of the tasks during the lesson. This feedback was very brief, less than 1 min, and very specific. For example, “You asked students to stop at the contact for drop shot, but you should have asked students to follow through after the contact,” or “For the high-skilled students, you might modify the tasks by asking them to hit the shuttles moving from the home position in the middle court as we discussed in the workshop.” Student Dependent Variables To validate the effectiveness of the teacher behaviors used in each condition, all trials during the instructional unit were measured for six students per class. Each of three observers per site observed every trial of two students during the 6-day units. Our dependent measures were created from a list of five critical elements for each of six primary skills that the teachers reported they planned to teach during the 6-day badminton units. Our initial selection of critical elements was derived from Grice (2008). Next, two badminton experts were asked to validate whether the selection of the critical elements of the skills as well as the criteria of the correct or incorrect trial were appropriate. The critical elements for the performance of these skills were organized into two or three phases of performance: preparation, execution, and follow-through, depending on the skill. Based on the representation by the teacher, the observer first determined whether the instructional focus of the task was directed to all phases or was specific to a phase (e.g., the preparation) of the skill. The performance of the student was then judged relative to meeting the presence or absence of critical elements. If all critical elements were present, the trial was coded as correct. If any of the critical elements were incorrectly performed, the trial was coded as incorrect. We used a third category called “other” to describe the following types of responses: (a) missed opportunity (i.e.,

134

P. WARD ET AL.

when a student missed hitting the shuttle due to the mistakes made by the student); (b) unfair opportunity (i.e., when a student missed hitting the shuttle due to an unhittable shuttle made by the partner such as the shuttle on the net, the shuttle landing on the outside boundary, or serves that were placed too long or too short making it unreasonable to execute the targeted shots); and (c) nontarget performance (i.e., when a student performed a skill or tactical movement that the teacher did not request). When any of these events occurred, they were coded as other.

Downloaded by [New York University] at 05:49 28 May 2015

Teacher-Dependent Variables Teacher performance data were coded using event recording (Cooper, Heron, & Heward, 2007). While watching the videotaped badminton lessons, the trained observers coded teacher data in terms of the appropriateness of task selections, maturity of task representations, and task adaptations. Appropriateness of the Tasks Teachers’ task appropriateness was coded with two categories: (a) developmental-appropriate (i.e., suitable for students’ age and individual capabilities), and principleappropriate (i.e., using fundamental principles of play practice). Using the combined categories, four combinations of developmentally appropriate and principle-appropriate tasks were categorized. First, a task was coded as both developmentally appropriate and principle-appropriate when a task was suitable for students’ age and skill level as well as underpinned fundamental principles of play practice (e.g., teaching alternative long and short serves using a service and return game to sixth graders). Second, a task was coded as developmentally appropriate and principle-inappropriate when a task was suitable for students’ age and skill level without using play practice principles (e.g., teaching a long-serve drill to eighth graders without specifically mentioning the target area). Third, a task was coded as developmentally inappropriate but principle-appropriate when a task was not suitable for students’ age and skill level but was used with fundamental principles of play practice (e.g., teaching a backhand drop shot on the first day of the unit to seventh graders using the lower net and large target). Fourth, a task was coded as both developmentally inappropriate and principle-inappropriate when the teacher provided a task that was not suitable for students’ age and skill level without using the fundamental principles of play practice (e.g., teaching an alternative stroke rally with a partner on the 1st day of the unit without changing the equipment or settings). Maturity of Task Representations Tasks were coded as mature or immature according to the following criteria. When a task included diverse types of visual and verbal representations such as cues, descriptions,

analogies, metaphors, or demonstrations that made it comprehensible to learners, it was coded as mature. For example, the following segment from an informing task statement (Rink, 2010) provided by a teacher is clear and concise: What we are going to do is to practice a forehand overhead stroke. Watch my demonstration. Both arms are up for the ready position. Turn your body for the side-hitting stance and scratch your back with your racket in preparation. Reach high to hit the shuttle using your wrist action making a huge rainbow shape. Aim to hit targets that are in the back boundary corners.

When a task lacked visual and/or verbal explanation of the critical elements of the skills, causing overly simplistic and/or incomplete representations of content, it was coded as immature. For example, the following informing task statement shows a lack of direction and clarity: “What we are going to do is to practice a forehand overhead stroke. I want to see you hit the shuttle to the targets that are in the back boundary corners.” Similarly if a task was described in overly complicated or verbose ways, it was coded as immature. Adaptations of Tasks Task adaptations were coded at two levels: (a) intertask adaptation (i.e., task development between tasks for the entire class), and (b) intratask adaptation (i.e., task development within tasks for small groups or individuals). For the intertask adaptations, the following four categories defined by Rink (2010) were used: (a) informing task (i.e., an initial task in the progression of a skill), (b) extending task (i.e., a task that changes the complexity or difficulty of student performance), (c) refining task (i.e., a task that focuses the quality of student performance), and (d) applying task (i.e., a task that changes the focus of learning from how to do the skill to how to use the skill in a game situation). For the intratask adaptations, the following four categories used by Ayvazo and Ward (2011) were utilized: (a) modifying task complexity (i.e., modifying space, equipment, number of students, or rules), (b) refining or breaking task (i.e., asking students to perform only one or two elements of the task for the quality of performance without changing the task), (c) restating task (i.e., repeating the entire task in forms other than the ones used when the task is delivered to the entire class), and (d) competition condition (i.e., changing a learning condition from noncompetitive to competitive situations). Data Collection Coder Training Six observers were trained at each site to collect data on student performances (total observers n ¼ 12). The training

EFFECTS OF IMPROVING TEACHERS’ CONTENT KNOWLEDGE

procedure for the observers was conducted in three phases:

135

. Level 4, different task and not consistent with

workshop from the teachers’ experimental classes.

Downloaded by [New York University] at 05:49 28 May 2015

. Phase 1: The observers learned the definitions for the

teacher and student variables. To check for their understanding, the observers took a written test. When the observers correctly answered at least 23 out of 26 questions relevant to the teacher variables and at least 9 out of 10 questions relevant to the student variables, they were able to move to the next phase. . Phase 2: The observers practiced coding variables while watching a 30-min training video. The training did not move forward to Phase 3 unless each observer met an 85% correct criterion. . Phase 3: Observers practiced live coding of student performances while watching 15-min episodes of badminton lessons. Observers were required to meet an 85% correct agreement criterion with a trained researcher. At this point, training was completed. The duration of observer training was approximately 10 hr at each site. Interobserver Agreement Interobserver agreement (IOA) was conducted on teacher and student variables for one third of all observations. IOA data for student variables were performed on 24 out of 72 randomly selected lesson observations per teacher per class. IOA for teacher variables resulted in observations of 8 of 24 lessons per teacher. The acceptable criterion for IOA was set at 85% (Cooper et al., 2007). The percentage of agreement was calculated using the following formula: Agreement divided by total trials (i.e., agreement plus disagreement) and then multiplied by 100. Overall, the mean IOAs obtained for both student and teacher data exceeded 88%, respectively (range ¼ 85.3% , 99.3% for students and range ¼ 78.9% , 92.3% for teachers). Treatment Integrity To maximize treatment integrity, the following strategies were used: (a) A video was created to standardize the presentation of information in the workshop; (b) a workshop checklist of key steps in the workshop was created to assess the extent to which the sites followed the same procedures; all sites reported 100% compliance; (c) a sample lesson plan was provided to show how tasks might be organized in a lesson; and (d) a teacher treatment checklist was used by researchers to code if the teachers implemented the tasks that were presented in the workshop as they taught their classes. Teacher treatment integrity data were recorded as one of four levels: . Level 1, as taught; . Level 2, partially correct; . Level 3, different task but consistent with workshop;

and

The data were calculated using the following formula: the number of tasks in each level divided by the total amount of tasks and then multiplied by 100 to compute the percentage of treatment integrity data. Overall, the four teachers’ treatment integrity at Level 1 met above a preset 80% criterion for the study with the mean of 88% (range ¼ 87.2% , 100%). Data Analysis For the analysis of student data, both descriptive and inferential statistics were used. The means, ranges, and percentage of students’ correct, incorrect, and other performances were analyzed descriptively. Analyses of variance (ANOVAs) with a nested structure were conducted to identify the statistical differences of the mean percentage of students’ correct, incorrect, and other performances between the comparison and experimental groups. Teachers were entered into the analyses as a block to eliminate the effects of teachers. Classes were nested within treatment conditions and teachers. The first statistical analysis consisted of one dependent variable of percentage of correct trials during a 6-day unit and eight independent variables: teacher, treatment, class nested within teacher and treatment, gender, skill levels, and three two-way interactions between treatment, gender, and skill levels. The second statistical analysis consisted of one dependent variable of percentage of incorrect trials during a 6-day unit and eight independent variables: teacher, treatment, class nested within teacher and treatment, gender, skill levels, and three two-way interactions between treatment, gender, and skill levels. For the analysis of teacher data, both descriptive and effect sizes were used. The means and ranges of teachers’ task representations, appropriateness, and adaptations were analyzed descriptively for both the comparison and the experimental classes. Cohen’s d (1988) was used to report effect sizes of teachers’ mature and immature task representations, developmental/principle-appropriate tasks, and intertask and intratask adaptations between the comparison and experimental classes. Data were analyzed using the SAS statistical software (SAS Institute Inc., 2008).

RESULTS How Does Student Achievement Differ as a Function of PCK? For the first statistical analysis using percentage of correct trials as a dependent variable, the Shapiro-Wilk’s test for normality showed a violation of normality assumption,

Downloaded by [New York University] at 05:49 28 May 2015

136

P. WARD ET AL.

W ¼ .96, p , .004. We obtained skewness (0.20) and kurtosis (2.03) values, which showed a moderate violation of normality. A square root transformation was conducted on percentage of correct trials for further data analysis. The Shapiro-Wilk’s test did not show a violation of normality assumption, W ¼ .98, p ¼ .29. The homogeneous variance tests for treatment conditions, gender, and skill levels did not show any violations of equality of variance. The first ANOVA showed that there were statistically significant differences in the percentage of correct trials for treatment conditions, F(1, 72) ¼ 310.73, p , .0001, h 2 ¼ .63 (Figure 1) and skill levels, F(2, 72) ¼ 29.69, p , .0001, h 2 ¼ .12. The analysis accounted for 85% of the total variance. Students in the experimental classes had a higher percentage of correct trials than did those in the comparison classes. Students with high and medium skills had a higher percentage of correct trials than did those with low skills. Students with high skills had a marginally higher percentage of correct trials than those with medium skills, but this finding was not statistically significant ( p ¼ .04) after a Bonferroni alpha adjustment (Type I error ¼ .02). In the second statistical analysis using percentage of incorrect trials as a dependent variable, the Shapiro-Wilk’s test for normality did not show any evidence of a violation of normality assumption, W ¼ .98, p ¼ .20. There was also no evidence of violations of homogeneous variance for treatment conditions and gender. There was marginal evidence showing a violation of homogeneous variance for skill levels, F(2, 93) ¼ 3.31, p , .04. No data transformation was conducted due to the rationale discussed earlier. The second ANOVA showed that there were statistically significant differences in the percentage of incorrect trials for treatment conditions, F(1, 72) ¼ 268.83, p , .0001, h 2 ¼ .67 (Figure 1), and skill levels, F(2, 72) ¼ 4.56, p , .01, h 2 ¼ .02. The analysis accounted for 82% of the total variance. Students in the comparison classes had a significantly higher percentage of incorrect trials than did

those in the experimental classes. Students with high skills had a marginally lower percentage of incorrect trials than did those with low skills. How Do Teachers’ Selections of Tasks Differ as a Function of CK? Table 1 shows the descriptive statistics (i.e., means and standard deviations) for teacher data from the 8 instructional units (i.e., four teachers £ two classes) in each treatment condition (total 16 units). The descriptive analysis of the teachers’ appropriate task selections per unit indicated that the teachers used both more developmentally appropriate and principle-appropriate tasks (Level 4) and only developmentally appropriate tasks (Level 3) in the experimental classes than in the comparison classes. In both comparison and experimental classes, only principle-appropriate tasks (Level 2) and both developmentally appropriate and principle-inappropriate tasks (Level 1) were rarely observed. The effect size for appropriate task selections was d ¼ 5.14, which exceeds Cohen’s (1988) convention for a large effect (d ¼ 0.80). How Do Teachers’ Representations of Tasks Differ as a Function of CK? The descriptive analysis of the teachers’ task maturity and the use of verbal and visual task representations showed that the teachers used more mature tasks and less immature tasks by using more diverse forms of verbal and visual task representations in the experimental classes than they did in the comparison classes (see Table 1). When we closely examined the forms of verbal task representations, the teachers’ uses of verbal descriptions, analogies/metaphors, cues, and specific congruent feedback per unit were increased while teaching the experimental classes after the badminton CK workshop. In terms of the

FIGURE 1 Mean percentage of correct and incorrect trials in comparison and experimental classes.

EFFECTS OF IMPROVING TEACHERS’ CONTENT KNOWLEDGE TABLE 1 Descriptive Statistics for Teacher Variables in Comparison and Experimental Classes Treatment Comparison

Downloaded by [New York University] at 05:49 28 May 2015

Dependent Variables Appropriate Task Selections Developmentally appropriate and principle-appropriate Only developmentally appropriate Only principle-appropriate Developmentally inappropriate and principle-inappropriate Mature Task Representations Maturity Immaturity Verbal Task Representations Instructions Descriptions Analogies and metaphors Cues Specific congruent feedback Visual Task Representations Full correct demonstration Partial correct demonstration Incorrect demonstration Visual aids Physical assistance Intertask Adaptations Informing Extending Refining Applying Intratask Adaptations Modifying Restating Refining and breaking Competitions condition

Experimental

M

SD

M

SD

4.3

1.3

16.5

3.1

12.6 0 1.5

4.3 0 1.7

3.6 0.4 0

4.7 0.5 0

2.0 16.3

3.0 4.0

18.3 2.1

3.9 2.2

41.4 17.9 2.4 25.1 35.9

20.3 11.3 2.2 16.2 22.6

69.5 63.9 16.4 71.5 112.1

14.5 29.8 8.9 16.8 34.5

1.5 4.5 30.6 1.3 6.5

3.5 5.3 26.4 2.1 7.0

93.4 11.4 3.4 1.1 14.4

31.1 14.0 4.0 2.1 12.4

7.5 5.8 1.9 3.4

1.7 2.8 2.2 2.3

9.5 5.8 6.6 3.4

2.3 2.6 4.0 1.9

1.4 2.8 0.6 0

1.8 0.7 0.7 0

4.9 19.7 22.8 0.1

3.4 10.4 19.7 0.4

forms of visual task representations, the means of full correct demonstrations, partial correct demonstrations, and physical assistances per unit in the experimental classes were higher than those in the comparison classes. However, the teachers’ incorrect demonstrations decreased after the CK workshop. In both comparison and experimental classes, the teachers rarely used visual teaching aids such as task cards, pictures, diagrams, or videos in teaching. Effect sizes were d ¼ 4.70 for task maturity and d ¼ 4.30 for immaturity. Both effect sizes exceed Cohen’s (1988) convention for a large effect (d ¼ 0.80).

How Do Teachers’ Adaptations of Tasks Differ as a Function of CK? The descriptive results of the teachers’ task adaptations indicated that the teachers used more intertask adaptations

137

for an entire class and used intratask adaptations for individuals or small groups throughout the badminton unit after the badminton CK workshop (see Table 1). When we examined the forms of intertask adaptations that the teachers used, the means of informing and refining tasks per unit in the experimental classes were higher than those in the comparison classes. On the other hand, the teachers used the same means of extending and applying tasks per unit in both comparison and experimental classes. In terms of the teachers’ use of intratask adaptations for small groups or individuals within the tasks, the means of modifying, restating, and refining/breaking tasks that the teacher used per unit in the experimental classes were higher than those in the comparison classes. Any intratask adaptations that are intended to change competition conditions for small groups or individuals were rarely observed in either comparison or experimental classes. The effect sizes were d ¼ 1.33 for intertask adaptations and d ¼ 2.02 for intratask adaptations. Both effect sizes exceed Cohen’s (1988) convention for a large effect (d ¼ 0.80).

DISCUSSION In this study, we examined the effects of a CK workshop intervention on the enacted PCK of teachers and in turn the effects of that PCK on student learning. We reported large effect size differences for teacher and student learning. In examining student learning, we used two statistical analyses with percentage of correct trials or percentage of incorrect trials as a dependent measure. Our data showed statistical significance for both analyses. More importantly, the effect sizes are .63 and .67, indicating a moderate-tohigh practical significance between groups in favor of the experimental condition (Cohen, 1988). Students in the experimental classes had a higher percentage of correct trials than those in the comparison classes. Conversely, students in the comparison group had a higher percentage of incorrect trials than those in the experimental group. These findings are important for three reasons. First, correct trials are highly associated with student achievement in physical education (Buck, Harrison, & Bryce, 1991; Hastie, Caldero´n, Palao, Ortega, 2011; Silverman, 1985). Second, in this study, we used criteria for correct trials that were independent of the teachers’ criteria to ensure that the criteria for a correct trial met a standard that was both valid and specific. Third, these results demonstrate significant gains in student learning. This is not a common outcome in physical education. In examining teacher PCK, we begin with Rink’s (1994, p. 276) observation that, “One might conclude that when the learner does not perform a skill correctly or appropriately, either the task was inappropriate or the task presentation was ineffective.” Task selection is indicative of SCK, while accurately describing the task requires knowledge of CCK.

Downloaded by [New York University] at 05:49 28 May 2015

138

P. WARD ET AL.

In this study, we hypothesized that a CK workshop that taught the CCK and SCK of badminton would impact teacher-enacted PCK exemplified in three classes of variables: a teacher’s (a) representation of tasks, (b) intertask development, and (c) intratask adaptation (Rink, 2010). We report very large effect sizes for all three variables in favor of the experimental group. In the comparison classes, teachers typically presented informing tasks with few if any extension and refining tasks. Research has consistently demonstrated that appropriate tasks differentiate between weak or strong PCK (Ayvazo & Ward, 2011; W. Chen & Rovegno, 2004; Rovegno, 1992, 1995; Schempp et al., 1998). Rovegno (1995) has pejoratively called this model of instruction a molecular view of task progression because of the disconnected scope and sequence of tasks and decontextualized teaching and learning. In contrast, the tasks used in the experimental classes were sequenced and successively more complex and challenging. Ward, Ayvazo, and Lehwald (2014) described this as a relational view of task progression and argued that this implies a set of fundamental principles that describe the connectivity of the content and how it should be developed. In this study, we called such connectivity principleappropriate. We concluded that in the experimental classes, the majority of the tasks were both developmentally appropriate and principle appropriate. In the comparison classes, there were few occasions when the task presented to the class as a whole was modified for different student needs. In the experimental classes, the teachers used more intratask adaptations with more restating, modifying, and refining tasks for the individuals or small groups. Ayvazo and Ward (2011) reported in their study that PCK was best evidenced by intratask adaptation by the teachers because this showed the depths of teacher knowledge, or what Shulman (1986) called the transformation of content. In previous studies of PCK, most conclusions about CK and PCK have been hypothetical and typically found in the discussion sections of articles. Our findings show that the enacted PCK of a teacher can be changed from immature to more mature as a function of CK and that this change can have a meaningful impact on student learning. The findings provided further evidence for Rovegno’s (1992) proposition that PCK varies on a continuum from weak to strong. Because the enacted PCK resulted in student learning and discriminated between conditions, our findings validate the three variables we used to measure a teacher’s enacted PCK. The strengths of this study include that this was an intervention study using a rigorous experimental design with a nesting structure where teachers were used as a block, classes were nested within conditions, and students were nested within classes. This nesting structure was incorporated into the statistical analyses with classes as an experimental unit (Silverman & Solmon, 1998). A second design strength lies in the measurement of the dependent variable for student learning. Many studies use the teacher’s

stated task criteria to the students as the criteria for the measurement of a dependent variable. In this study, the critical elements of technique and tactics that served as the dependent variables were established prior to the start of the study and were used as independently validated criteria. A third design strength is that we collected data on every single performance of the six students in each class during the 6-day unit. Thus, although the number of students sampled was small, we report a complete picture of student learning for each day of the unit. In terms of the results of the study, the effect sizes reported in this study for student learning are unusually robust and show a strong effect of teachers’ PCK on student performance. Similarly the effect sizes for teachers are very large and show a strong effect of teachers’ CK on PCK. A final strength of this study is that these gains were obtained from an intervention workshop that was 4 hr in duration. The implication for future research is to examine more closely the strategies used in the workshop and, most importantly, in our view, the content of the workshop focusing on SCK. The limitations of our study provide directions for future research. First, the strategy of having teachers teach control and experimental classes is good. Data contamination was avoided by having teachers teach control classes first, then attend the workshop, and lastly teach the experimental classes. However, the classes were not randomly assigned into conditions and the numbers of teachers and students were small. This limitation makes it difficult to generalize the findings of this study. Future studies might focus on increasing the number of teachers and students and randomizing the classes. Second, we did not use preworkshop – postworkshop measures of teacher knowledge. We relied instead on (a) the 96% criterion for completing workshop tasks, and (b) the fidelity of the teacher-enacted PCK to demonstrate evidence of change. To our knowledge, valid and reliable knowledge tests of SCK do not exist at present. Measures of CK and in particular SCK are needed to demonstrate direct changes in teacher knowledge.

WHAT DOES THIS ARTICLE ADD? Because experience has a weak correlation to teaching effectiveness and in particular CK, policymakers must rely on preservice and continuing education to prepare teachers to teach CK. This study contributes to the literature in three ways. First, it is the first experimental study demonstrating the efficacy of a CK intervention on the enacted PCK of teachers and on student learning. Second, it is among the first studies in physical education to unpack CK in terms of CCK and SCK and to create a professional development workshop that specifically teaches these elements. Third, the study is among the first to operationally define and measure PCK using the teachers’ (a) selections of tasks, (b)

EFFECTS OF IMPROVING TEACHERS’ CONTENT KNOWLEDGE

representations of tasks, and (c) adaptations of tasks as measures of PCK.

Downloaded by [New York University] at 05:49 28 May 2015

REFERENCES Abell, S. K. (2008). Twenty years later: Does pedagogical content knowledge remain a useful idea? International Journal of Science Education, 30, 1405–1416. Ayvazo, S., & Ward, P. (2011). Pedagogical content knowledge of experienced teachers in physical education: Functional analysis of adaptations. Research Quarterly for Exercise and Sport, 82, 675–684. Ball, D. L., Thames, M. H., & Phelps, G. (2008). Content knowledge for teaching: What makes it special? Journal of Teacher Education, 59, 389–407. Buck, M., Harrison, J., & Bryce, G. (1991). An analysis of learning trials and their relationship to achievement in volleyball. Journal of Teaching in Physical Education, 10, 134 –152. Chen, A., & Ennis, C. D. (1995). Content knowledge transformation: An examination of the relationship between content knowledge and curriculum. Teaching and Teacher Education, 11, 389–402. Chen, W., & Rovegno, I. (2004). Learning the skill theme approach: Salient and problematic aspects of pedagogical content knowledge. Education, 124, 194 –212. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis. Upper Saddle River, NJ: Pearson-Prentice Hall. Grice, T. (2008). Badminton steps to success. Champaign, IL: Human Kinetics. Grossman, P. L. (1990). The making of a teacher: Teacher knowledge and teacher education. New York, NY: Teacher College Press. Hastie, P. A., Caldero´n, A., Palao, J., & Ortega, E. (2011). Quantity and quality of practice: Interrelationships between task organization and student skill level in physical education. Research Quarterly for Exercise and Sport, 82, 784–787. Hastie, P., & Vlaisavljevic, N. (1999). The relationship between subjectmatter expertise and accountability in instructional tasks. Journal of Teaching in Physical Education, 19, 22–33. Jenkins, J. M., & Veal, M. L. (2002). Preservice teachers’ PCK development during peer coaching. Journal of Teaching in Physical Education, 22, 49–68.

139

Launder, A. (2001). Play practice: The games approach to teaching and coaching sports. Champaign, IL: Human Kinetics. Marks, R. (1990). Pedagogical content knowledge: From a mathematical case to a modified conception. Journal of Teacher Education, 41, 3 –11. McCaughtry, N., & Rovegno, I. (2003). Development of pedagogical content knowledge: Moving from blaming students to predicting skillfulness, recognizing motor development and understanding emotion. Journal of Teaching in Physical Education, 22, 355–368. Rink, J. (1994). Task presentation in pedagogy. Quest, 46, 270–280. Rink, J. (2010). Teaching physical education for learning (6th ed.). New York, NY: McGraw Hill. Rovegno, I. (1992). Learning to teach in a field-based method course: The development of pedagogical content knowledge. Teaching and Teacher Education, 8, 69–82. Rovegno, I. (1995). Theoretical perspectives on knowledge and learning and a student teacher’s pedagogical content knowledge of dividing and sequencing subject matter. Journal of Teaching in Physical Education, 14, 284 –304. SAS Institute Inc. (2008). SAS/STAT w 9.2 User’s guide. Cary, NC: Author. Schempp, P., Manross, D., Tan, S., & Fincher, M. (1998). Subject expertise and teacher’s knowledge. Journal of Teaching in Physical Education, 17, 342–356. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15, 4–14. Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, 1–22. Silverman, S. (1985). Relationship of engagement and practice trials to student achievement. Journal of Teaching in Physical Education, 5, 13–21. Silverman, S., & Solmon, M. (1998). The unit of analysis in field research: Issues and approaches to design and data analysis. Journal of Teaching in Physical Education, 17, 270–284. Ward, P. (2009). Content matters: Knowledge that alters teaching. In L. D. Housner, M. W. Metzler, P. Schempp, & T. Templin (Eds.), Historic traditions and future directions of research on teaching and teacher education in physical education (pp. 345–356). Morgantown, WV: Fitness Information Technology. Ward, P., Ayvazo, S., & Lehwald, H. (2014). Using knowledge packets in teacher education to develop pedagogical content knowledge. Journal of Physical Education, Health, Recreation & Dance, 85(6), 38 – 43.

Effects of improving teachers' content knowledge on teaching and student learning in physical education.

The purpose of this study was to examine the efficacy of a content knowledge (CK) workshop on the enacted pedagogical content knowledge (PCK) of teach...
147KB Sizes 0 Downloads 4 Views