Direct Instruction in Math Word Problems: Students With Learning Disabilities CYNTHIA L. WILSON University ofMiami PAUL T. SINDELAR University ofFlorida
ABSTRACT: This investigation was conducted to test the effectiveness of strategy teaching and sequencing practice problems in teaching students with learning disabilities to identify the correct algorithm for solving addition and subtraction word problems. Sixty-two students were assigned to one ofthree experimental groups: strategy plus sequence, strategy only, and sequence only. The results indicated that students in the strategy-plus-sequence group, as well as those in the strategy-only group, scored significantly higher than did students in the sequence-only group. Findings indicated that strategy teaching was the more effective of the two instructional components. Implications are discussed in terms ofinstructional design for students with learning disabilities.
D Everyday acts such as deciding whether one can afford to purchase an item require the application of problem-solving skills. Because students with mild disabilities will live independent and productive lives, problem-solving skills are as essential for them as for students without disabilities. However, students with disabilities are less likely to adopt a strategic approach to problem solving (Torgesen & Kail, 1980); thus, they are likely to experience difficulty in mastering the skill. It is crucial that the mathematics program for these students include problem solving as a part of the curriculum. An examination of elementary school mathematics curricula has Shown that the design of commonly used basal arithmetic programs may largely be responsible for the difficulty students have in solving word problems (Silbert, Carnine, & Stein, 1981). Major concerns about basal programs include a lack of adequate provision for practice and review, inadequate sequencing of problems, and an absence of strategy teaching and step-by-step procedures for teaching word problem solving. Mathematics curricula of students with mental disabilities also lack adequate, strategic, and se-
quenced instruction (Cawley, Fitzmaurice, Shaw, Kahan, & Bates, 1978, 1979a, 1979b; Cawley & Goodman, 1968, 1969; Englemann, 1977). In fact, Englemann argued that it is inadequate instruction rather than deficits in cognitive processes that accounts for the greatest proportion of failure of students to learn basic academic skills. Furthermore, for students with mild disabilities, the curricula too often emphasize rote development of computational skills (Cawley et aI., 1978, 1979a, 1979b; Cawley & Goodman, 1968, 1969). A review of the literature specific to math and students with learning disabilities revealed that only "scant attention" is directed toward the word-problem-solving skills of these students (Cawley, Miller, & School, 1987, p. 87). The little research that exists suggests that consideration must be given to sequencing, providing adequate practice, cognitive strategies, direct instruction, and techniques to promote generalization (Darch, Carnine, & Gersten, 1984; Fleischner & O'Loughlin, 1985; Jones, Krouse, Feorene, & Saferstein, 1985; Montague & Bos, 1986). Two studies (Darch et al., 1984; Jones et al., 1985) evaluated the effectiveness of direct inMay 1991
512
Downloaded from ecx.sagepub.com at Bobst Library, New York University on January 26, 2016
struction. Darch et al. compared the effectiveness of a direct-instruction approach to that of a basalmath approach for teaching fourth graders without disabilities to solve word problems. The results indicated that students who were taught using direct instruction performed significantly higher on the posttest than did students who were taught by more traditional methods. This study focused on the effects of sequencing problem types and using a direct-instruction strategy for problem solving. This study sought to build on a study by Jones et al. (1985), who compared two variations of a direct-instruction strategy for teaching students without disabilities to solve addition and subtraction word problems. In both variations, the "big number" concept (Silbert, Carnine, & Stein, 1981) was taught. With it, students determined whether a problem gives the big number of a fact family. If it does, the problem requires subtraction; if not, the problem requires addition. This approach calls for direct teaching of articulated strategies for translation of word problems into equations. In the sequential variation, students practiced solving word problems sequenced according to type; in the concurrent variation, students practiced a balanced combination of problem types. Jones et al. found that students in the sequential condition made significantly greater gains over the 9-day instructional period than did the concurrent group. However, because only students who scored at or below 25% on the pretest were included in the study and the mean pretest score of the sequential group was significantly lower than the mean pretest score of the concurrent group, some portion of the posttest differences may be attributable to statistical regression to the mean. Moreover, Jones et al. attributed their findings to the sequencing of practice problems. They asserted that teachers "using other teaching methods should consider the advantages of sequenced introduction in teaching students to solve verbal math problems" (p. 29), even though they did not isolate the effects of sequencing from the directinstruction approach. Finally, whether the findings of this study would hold for a population of students with learning disabilities is also uncertain. The purpose of this study was to compare the effectiveness of the three procedures for teaching students with learning disabilities to identify the correct algorithm in solving addition and subtraction word problems. In the process, the study per-
mitted an analysis of the effectiveness of the components (strategy teaching and sequencing) of the instructional package, and extended to the exceptional population results already established for students without disabilities. It was hypothesized that students in the strategy-plussequence group would score significantly higher on both the posttest and the follow-up test than those students in the strategy-only and sequenceonly groups. METHOD
Subjects The subjects who participated in this study were 62 students with learning disabilities (LD) from nine elementary schools in a medium-sized school district in north Florida. To participate in the study, students had to meet the following criteria: (a) they were labeled LD according to district criteria, (b) they attended a special education math program, (c) they scored at 80% or better on a test of basic addition and subtraction skills, (d) they read on at least a 1.5 grade level, and (e) they were identified by their LD teacher as needing instruction in word problem solving. Descriptive information on the students composing the three groups are presented in Table 1. Included are mean age, IQ tests, and achievement test scores, as well as distributions on grade placement, race, sex, and special education placement. The groups did not differ significantly on mean IQ and proportions of males and females. The students in the strategy-plus-sequence group, however, were younger, were placed in lower grades, and scored lower on both reading and math achievement tests, These differences militated against our finding significant effects for the strategy-plus-sequence group. Minorities were underrepresented in the strategy-only group and over- represented in the strategy-plus-sequence group. In the sequence-only group, a greater proportion of students was placed in full-time programs. Instructional groups of three to five students were formed within schools on the basis of student schedules. Because there were often two or more instructional groups at one school, assignment of groups to treatment was done by schools. In this way, within-school contamination of the treatment was avoided. To control for the effect of socioeconomic status (SES), a stratified random assignment procedure was used. Each treat-
Exceptional Children
513
Downloaded from ecx.sagepub.com at Bobst Library, New York University on January 26, 2016
TABLE 1 Comparison of Treatment Groups on Pretreatment Data Pretreatment Data
Strategy Plus Sequence
Strategy Only
Sequence Only
9-7 89.5
10-10 96.7
10-2 94.5
2.0 2.4
3.1 3.6
2.6 3.1
Average age (years) Average IQ Average Woodcock-Johnson score: Reading Math Grade level (n) 2nd and 3rd 4th and 5th Racial designation (n) White Nonwhite Sex (n) Male Female Special education placement (n) Part time Full time
ment group comprised instructional groups from one low-, one middle-, and one high-SES school.
Setting Students were taught in groups of three to five. Instruction took place in a number of settings, including the students' LD classroom, office space, the media center, and the cafeteria. During instruction, students sat in a semicircle facing the experimental trainer and away from any other students in the room. Each lesson lasted approximately 30 minutes (min) for all groups.
Materials The 216 word problems used in the pretest, posttest, and follow-up test and training materials were selected from first- and second-grade math materials and consisted of two or three sentences. The problems were divided into four main types, as described by Silbert et al. (1981): simple action problems, classification problems, complex action problems, and comparison problems. The four problem types were equally represented in the pool of word problems. As in the Jones et al. (1985) study, two types of materials were developed: boardwork problems and seatwork problems. Four boardwork
17 4
5 16
12
6 15
14 7
8 12
18 3
15 6
19 1
17 4
20 1
9 11
8
problems were presented each day (for 12 days) for all three groups. The students in the strategyplus-sequence and the sequence-only groups received one type of boardwork problem per day for 3-day periods. The order of the presentation of problem type was simple action, classification, complex action, and comparison. The strategyonly group received a mixture of problem types presented in no particular order. They also received a total of 12 examples for each problem type over the course of the lessons. In fact, the same training problems were used in all three conditions, although the problems were not presented in the same order. Seatwork problems were used for individual practice. For the strategy-plus-sequence and sequence-only groups, 24 seatwork problems were selected for each problem type. Six seatwork items were practiced on the first day, 8 on the second, and 10 on the third. The strategy-only group also completed 6, 8, and 10 seatwork problems in recurring 3-day cycles; but each day's worksheet included a balance of the four problem types. At the end of the study, students in all three groups had practiced during seatwork the same set of 96 problems. The students were administered a two-part pretest. The first part was the test of basic addi-
514
May 1991
Downloaded from ecx.sagepub.com at Bobst Library, New York University on January 26, 2016
TABLE 2 Obtained and Adjusted Means and Standard Deviations for the Three Experimental Groups Test Data No. of students Pretest Mean SD Posttest Obtained M SD Adjusted M Follow-up test ObtainedM SD Adjusted M
Strategy Plus Sequence
Strategy Only
Sequence Only
21
21
20
10.95 3.72
16.67 1.19
13.85 2.93
14.95 2.52 16.42
17.29 2.97 15.83
14.00 3.35 13.99
16.05 2.75 17.48
16.43 2.71 15.00
12.95 3.10 12.94
tion and subtraction facts used to determine eligibility for the study. The test comprised randomly selected samples of 25 addition problems (with single-digit addends and sums of 18 or less) and 25 subtraction problems (with single-digit subtrahends and differences of 9 or less). Of the 78 students who took the basic-facts test, 62 met the criteria and participated in the study. The second part of the pretest consisted of 24 word problems, including 3 addition and 3 subtraction problems from each of the four problem types. The posttest and follow-up test were equivalent forms of the word-problem part of the pretest. The posttest was administered to all students at the end of the 3-week instructional period and the follow-up test was administered 2 weeks after the conclusion of the study. On all three of the tests, students were assessed on the ability to write the correct algorithm but were not scored on the accuracy of computing answers. This was consistent with the methodology of Darch et aI. and Jones et al., and with the literature that has shown that computational skills account for a small proportion of the variance in solving story problems (Muth, 1984). Students were given as much time as necessary to complete the tests; most completed them in 30 min. To determine reliability of instruments, all three forms were administered to 30 third graders. Indexes of internal consistency were computed using the Kuder-Richardson formula 20. The results were .86, .88, and .85 for the pretest, posttest, and follow-up test, respectively. To demonstrate equivalency of the three tests, mean scores were calculated. The scores on the pretest
averaged 80% correct, whereas the posttest and follow-up scores averaged 81% and 83% correct. Data for all three tests are presented in Table 2.
Trainer Instruction The seven trainers who participated in the study were students who were majoring in special education and who had some classroom experience. Five trainers instructed more than one group of students; however, each trainer taught only one kind oflesson. Trainers attended five l-hour sessions over a 2-week period before the beginning of the study. The assignment of trainers to groups was made on the basis of the times that they were available for instruction. The training for each treatment occurred in the same manner, but was conducted separately for the trainers in each treatment. Using role-playing techniques, the experimenter demonstrated the lessons for each of the treatment conditions. Following the demonstrations, each person was given a chance to act out the part of the instructor while others acted as observers. Trainers were required to score 80% or above on three consecutive demonstrations before they could participate in the study.
Procedural Reliability To determine the fidelity with which the lessons were presented, each trainer was observed twice during the course of the study. Observers, who also were undergraduate special education majors, were trained to complete an observational 515
Exceptional Children
Downloaded from ecx.sagepub.com at Bobst Library, New York University on January 26, 2016
protocol with which trainers' presentations could be evaluated. The presentations of trainers in the strategy-plus-sequence and strategy-only conditions (i.e., trainers who taught the big number concept) were judged on the presence or absence of six elements of their scripted lessons; trainers in the sequence-only condition were judged on seven elements. The criterion for reliable administration of the lessons was at least 80% "yes" responses. The reliability scores averaged 89% and ranged from 83% to 97%. No trainer ever fell below the 80% criterion and required reteaching. Treatment Similarities Although the students in the three treatment groups were taught with different instructional methods, the treatments had several features in common. In all training conditions, groups of three to five students were engaged in instruction for 14 30-min sessions. Instruction occurred Monday through Friday for 3 weeks. Another similarity was the use of detailed, scripted lesson plans. All groups received the same number and identical sets of word problems in both the boardwork and seatwork phases of the lessons. During seatwork, trainers were available to answer any questions students had; and they reinforced students for accurate work and appropriate social behavior. Strategy Only and Strategy Plus Sequence Fourteen 30-min lessons were developed for both treatments according to the recommendations of Silbert et al. (1981, chapter 12). Students responded in unison to questions. Hand signals were used to control responding, as well as to ensure that all students in the group were participating. During the first 2 days of training, the groups were given instruction on the fact-family concept. This instruction was presented in three ways. First, the rule about what to do when the big number was given was taught. For example, the students were told that "When the big number is given, subtract." After telling students the rule, the trainer demonstrated its application, writing a diagram on the board in which the big number was given. Next, the trainer introduced the rule about what to do when the big number was not given. This application was also diagrammed on the board. After approximately 15 min of boardwork practice, students were given seatwork practice sheets.
On Day 3, students were taught to apply the big number concept to solving word problems. Although both the strategy-plus-sequence and the strategy-only groups received the same strategy instruction, the presentation of practice problems differed. The strategy-plus-sequence groups received all of the same kind of practice problems. The first three lessons involved simple action problems, followed by three lessons of classification problems, three lessons of complex action problems, and three lessons of comparison problems. The strategy-only group received a balanced combination of practice problems that included all four types each day. Sequence Only The 14 30-min lessons that comprised the sequence-only group were adopted from Mathematics Today (1985), the basal math series used in the students' schools. The first 2 lessons involved practice in the computation of addition and subtraction problems. On Day 3, the instruction on word problems began. The boardwork portion of the lessons involved the use of a questioning technique that prompted students to see information in the problem. A typical question might be, "What numbers are given in the problem?" or "What are we supposed to find?" As the children answered the questions, the trainer wrote their answers on the board. Once boardwork was completed, students were assigned seatwork problems. The students in this group practiced both boardwork and seatwork problems that had been grouped according to problem type in the same manner as the strategy-plus-sequence group. That is, the first three lessons involved simple action problems, followed by 3 days of classification problems, 3 days of complex action problems, and 3 days of comparison problems. RESULTS Data were analyzed to determine if the independent variable of type of instruction (strategy plus sequence, strategy only, sequence only) affected the dependent variable of number of problems solved correctly. A problem was counted correct if the correct algorithm was used in computing the answer. Analysis of covariance (ANCOVA) was used, with the pretest serving as the covariate. Separate ANCOVAs were conducted on the posttest and the follow-up tests. Students rather
516
May 1991
Downloaded from ecx.sagepub.com at Bobst Library, New York University on January 26, 2016
than group served as the unit in these analyses because the use of trained experimenters, scripted lessons, and prescribed examples minimized variation due to teachers. Table 2 contains the obtained and adjusted means and standard deviations for the three experimental groups. The obtained means were adjusted statistically to take into account the initial differences among the pretest means. The adjusted means represent the best possible estimates as to what the groups would have scored on the posttest and follOW-Up measures if the three groups had started with identical pretest means. On the analysis of the posttest data, a significant effect for treatment was found, F(2, 58) = 4.69,p = .013. To determine specifically how the treatments differed, post hoc analyses were conducted using the adjusted means for each treatment, according to the procedure described by Pedhazur (1982) for ANCOVA. Although students who received instruction in the strategy-plus-sequence group scored higher than did students in both the strategy-only and sequence-only groups, the post hoc analysis indicated that the strategy-plus-sequence group differed significantly only from the sequenceonly group, F(1, 57) = 7.49,p < .05. Further, students in the strategy-only group also scored significantly higher on the average than did students in the sequence-only groups, F(1, 57) = 4.33,p < .05. The ANCOVA for the follow-up test also yielded a significant effect for treatment, F(2, 58) = l4.71,p= .0001. The post hoc analysis revealed that the adjusted mean score of the strategy-plussequence group was significantly higher than the adjusted mean scores of the sequence-only and strategy-only group. Furthermore, the strategyonly group scored significantly higher than did the sequence-only group. An analysis was undertaken using unadjusted means to describe the performance of the LD students, taken as a whole, relative to the third-grade normative sample. These results are presented in Figure 1. On the pretest, the LD students solved 5.41 fewer problems than did the third graders; on the posttest, this difference decreased to 3.96. Thus, on the average, the LD students in this study approached, but never reached, the level of performance of the third graders despite the fact that all had participated in 14 lessons on solving word problems. On the other hand, the pretest to posttest improvement for the LD students (M = 1.61) was statistically significant, 1(61) = 3.77, p
< .001, despite the fact that the effects of the three treatments were averaged. DISCUSSION The analysis of the data indicated significant differences among the groups on both the posttest and the follow-up test. Students in the strategyplus-sequence group scored significantly higher on both measures than did students in the sequence-only group. Students in the strategy-only group also scored significantly higher on both the posttest and the follow-up test than did students in the sequence-only group. However, on the posttest, the difference between the strategyplus-sequence and strategy-only groups was not significant; on the follow-up test, the strategyplus-sequence group scored significantly higher. One explanation for this difference may be the continued use of the big number concept by a classroom aide, unbeknownst to the experimenters. The aide picked up on the strategy and continued its use with a strategy-plus-sequence group. Thus to conclude that strategy plus sequence produced a more enduring effect than strategy only would be an arguable point. Nonetheless, the data indicated the superiority of a direct-instruction approach for teaching students with learning disabilities to solve addition and subtraction word problems, regardless of whether that direct instruction approach incorporated sequenced examples. Our results were consistent with those of Darch et al. (1984), but not with Jones et al. (1985). In the present study, as well as that of the Darch et al., the direct-instruction approach led to superior performance when compared with a basal approach. It should be pointed out that both studies used the big number concept for strategy teaching and that the basal approach in the Darch et al. study was similar to that used in the present study. In this study, we sought to refine the findings of Jones et al. (1985), who also examined the effects of sequencing problems according to type versus teaching all problem types concurrently. In the present study, however, the design permitted an assessment of the effects of strategy teaching and sequencing in isolation. The comparison of the strategy-plus-sequence and strategy-only groups represented an exact replication of the Jones et al. (1985) study; however, Jones et al. reported that students who received strategy-plus-sequence instruction scored higher than did those in the strategy-only group.
Exceptional Children
517
Downloaded from ecx.sagepub.com at Bobst Library, New York University on January 26, 2016
FIGUREl Pretest and Posttest Performance of the Two Samples: Students With Learning Disabilities (LD) and Third Graders Without Disabilities
25.,----------------------,
3rd Graders
20
LD
III
c
co
ABSTRACT: This investigation was conducted to test the effectiveness of strategy teaching and sequencing practice problems in teaching students with learning disabilities to identify the correct algorithm for solving addition and subtraction word problems. Sixty-two students were assigned to one ofthree experimental groups: strategy plus sequence, strategy only, and sequence only. The results indicated that students in the strategy-plus-sequence group, as well as those in the strategy-only group, scored significantly higher than did students in the sequence-only group. Findings indicated that strategy teaching was the more effective of the two instructional components. Implications are discussed in terms ofinstructional design for students with learning disabilities.
D Everyday acts such as deciding whether one can afford to purchase an item require the application of problem-solving skills. Because students with mild disabilities will live independent and productive lives, problem-solving skills are as essential for them as for students without disabilities. However, students with disabilities are less likely to adopt a strategic approach to problem solving (Torgesen & Kail, 1980); thus, they are likely to experience difficulty in mastering the skill. It is crucial that the mathematics program for these students include problem solving as a part of the curriculum. An examination of elementary school mathematics curricula has Shown that the design of commonly used basal arithmetic programs may largely be responsible for the difficulty students have in solving word problems (Silbert, Carnine, & Stein, 1981). Major concerns about basal programs include a lack of adequate provision for practice and review, inadequate sequencing of problems, and an absence of strategy teaching and step-by-step procedures for teaching word problem solving. Mathematics curricula of students with mental disabilities also lack adequate, strategic, and se-
quenced instruction (Cawley, Fitzmaurice, Shaw, Kahan, & Bates, 1978, 1979a, 1979b; Cawley & Goodman, 1968, 1969; Englemann, 1977). In fact, Englemann argued that it is inadequate instruction rather than deficits in cognitive processes that accounts for the greatest proportion of failure of students to learn basic academic skills. Furthermore, for students with mild disabilities, the curricula too often emphasize rote development of computational skills (Cawley et aI., 1978, 1979a, 1979b; Cawley & Goodman, 1968, 1969). A review of the literature specific to math and students with learning disabilities revealed that only "scant attention" is directed toward the word-problem-solving skills of these students (Cawley, Miller, & School, 1987, p. 87). The little research that exists suggests that consideration must be given to sequencing, providing adequate practice, cognitive strategies, direct instruction, and techniques to promote generalization (Darch, Carnine, & Gersten, 1984; Fleischner & O'Loughlin, 1985; Jones, Krouse, Feorene, & Saferstein, 1985; Montague & Bos, 1986). Two studies (Darch et al., 1984; Jones et al., 1985) evaluated the effectiveness of direct inMay 1991
512
Downloaded from ecx.sagepub.com at Bobst Library, New York University on January 26, 2016
struction. Darch et al. compared the effectiveness of a direct-instruction approach to that of a basalmath approach for teaching fourth graders without disabilities to solve word problems. The results indicated that students who were taught using direct instruction performed significantly higher on the posttest than did students who were taught by more traditional methods. This study focused on the effects of sequencing problem types and using a direct-instruction strategy for problem solving. This study sought to build on a study by Jones et al. (1985), who compared two variations of a direct-instruction strategy for teaching students without disabilities to solve addition and subtraction word problems. In both variations, the "big number" concept (Silbert, Carnine, & Stein, 1981) was taught. With it, students determined whether a problem gives the big number of a fact family. If it does, the problem requires subtraction; if not, the problem requires addition. This approach calls for direct teaching of articulated strategies for translation of word problems into equations. In the sequential variation, students practiced solving word problems sequenced according to type; in the concurrent variation, students practiced a balanced combination of problem types. Jones et al. found that students in the sequential condition made significantly greater gains over the 9-day instructional period than did the concurrent group. However, because only students who scored at or below 25% on the pretest were included in the study and the mean pretest score of the sequential group was significantly lower than the mean pretest score of the concurrent group, some portion of the posttest differences may be attributable to statistical regression to the mean. Moreover, Jones et al. attributed their findings to the sequencing of practice problems. They asserted that teachers "using other teaching methods should consider the advantages of sequenced introduction in teaching students to solve verbal math problems" (p. 29), even though they did not isolate the effects of sequencing from the directinstruction approach. Finally, whether the findings of this study would hold for a population of students with learning disabilities is also uncertain. The purpose of this study was to compare the effectiveness of the three procedures for teaching students with learning disabilities to identify the correct algorithm in solving addition and subtraction word problems. In the process, the study per-
mitted an analysis of the effectiveness of the components (strategy teaching and sequencing) of the instructional package, and extended to the exceptional population results already established for students without disabilities. It was hypothesized that students in the strategy-plussequence group would score significantly higher on both the posttest and the follow-up test than those students in the strategy-only and sequenceonly groups. METHOD
Subjects The subjects who participated in this study were 62 students with learning disabilities (LD) from nine elementary schools in a medium-sized school district in north Florida. To participate in the study, students had to meet the following criteria: (a) they were labeled LD according to district criteria, (b) they attended a special education math program, (c) they scored at 80% or better on a test of basic addition and subtraction skills, (d) they read on at least a 1.5 grade level, and (e) they were identified by their LD teacher as needing instruction in word problem solving. Descriptive information on the students composing the three groups are presented in Table 1. Included are mean age, IQ tests, and achievement test scores, as well as distributions on grade placement, race, sex, and special education placement. The groups did not differ significantly on mean IQ and proportions of males and females. The students in the strategy-plus-sequence group, however, were younger, were placed in lower grades, and scored lower on both reading and math achievement tests, These differences militated against our finding significant effects for the strategy-plus-sequence group. Minorities were underrepresented in the strategy-only group and over- represented in the strategy-plus-sequence group. In the sequence-only group, a greater proportion of students was placed in full-time programs. Instructional groups of three to five students were formed within schools on the basis of student schedules. Because there were often two or more instructional groups at one school, assignment of groups to treatment was done by schools. In this way, within-school contamination of the treatment was avoided. To control for the effect of socioeconomic status (SES), a stratified random assignment procedure was used. Each treat-
Exceptional Children
513
Downloaded from ecx.sagepub.com at Bobst Library, New York University on January 26, 2016
TABLE 1 Comparison of Treatment Groups on Pretreatment Data Pretreatment Data
Strategy Plus Sequence
Strategy Only
Sequence Only
9-7 89.5
10-10 96.7
10-2 94.5
2.0 2.4
3.1 3.6
2.6 3.1
Average age (years) Average IQ Average Woodcock-Johnson score: Reading Math Grade level (n) 2nd and 3rd 4th and 5th Racial designation (n) White Nonwhite Sex (n) Male Female Special education placement (n) Part time Full time
ment group comprised instructional groups from one low-, one middle-, and one high-SES school.
Setting Students were taught in groups of three to five. Instruction took place in a number of settings, including the students' LD classroom, office space, the media center, and the cafeteria. During instruction, students sat in a semicircle facing the experimental trainer and away from any other students in the room. Each lesson lasted approximately 30 minutes (min) for all groups.
Materials The 216 word problems used in the pretest, posttest, and follow-up test and training materials were selected from first- and second-grade math materials and consisted of two or three sentences. The problems were divided into four main types, as described by Silbert et al. (1981): simple action problems, classification problems, complex action problems, and comparison problems. The four problem types were equally represented in the pool of word problems. As in the Jones et al. (1985) study, two types of materials were developed: boardwork problems and seatwork problems. Four boardwork
17 4
5 16
12
6 15
14 7
8 12
18 3
15 6
19 1
17 4
20 1
9 11
8
problems were presented each day (for 12 days) for all three groups. The students in the strategyplus-sequence and the sequence-only groups received one type of boardwork problem per day for 3-day periods. The order of the presentation of problem type was simple action, classification, complex action, and comparison. The strategyonly group received a mixture of problem types presented in no particular order. They also received a total of 12 examples for each problem type over the course of the lessons. In fact, the same training problems were used in all three conditions, although the problems were not presented in the same order. Seatwork problems were used for individual practice. For the strategy-plus-sequence and sequence-only groups, 24 seatwork problems were selected for each problem type. Six seatwork items were practiced on the first day, 8 on the second, and 10 on the third. The strategy-only group also completed 6, 8, and 10 seatwork problems in recurring 3-day cycles; but each day's worksheet included a balance of the four problem types. At the end of the study, students in all three groups had practiced during seatwork the same set of 96 problems. The students were administered a two-part pretest. The first part was the test of basic addi-
514
May 1991
Downloaded from ecx.sagepub.com at Bobst Library, New York University on January 26, 2016
TABLE 2 Obtained and Adjusted Means and Standard Deviations for the Three Experimental Groups Test Data No. of students Pretest Mean SD Posttest Obtained M SD Adjusted M Follow-up test ObtainedM SD Adjusted M
Strategy Plus Sequence
Strategy Only
Sequence Only
21
21
20
10.95 3.72
16.67 1.19
13.85 2.93
14.95 2.52 16.42
17.29 2.97 15.83
14.00 3.35 13.99
16.05 2.75 17.48
16.43 2.71 15.00
12.95 3.10 12.94
tion and subtraction facts used to determine eligibility for the study. The test comprised randomly selected samples of 25 addition problems (with single-digit addends and sums of 18 or less) and 25 subtraction problems (with single-digit subtrahends and differences of 9 or less). Of the 78 students who took the basic-facts test, 62 met the criteria and participated in the study. The second part of the pretest consisted of 24 word problems, including 3 addition and 3 subtraction problems from each of the four problem types. The posttest and follow-up test were equivalent forms of the word-problem part of the pretest. The posttest was administered to all students at the end of the 3-week instructional period and the follow-up test was administered 2 weeks after the conclusion of the study. On all three of the tests, students were assessed on the ability to write the correct algorithm but were not scored on the accuracy of computing answers. This was consistent with the methodology of Darch et aI. and Jones et al., and with the literature that has shown that computational skills account for a small proportion of the variance in solving story problems (Muth, 1984). Students were given as much time as necessary to complete the tests; most completed them in 30 min. To determine reliability of instruments, all three forms were administered to 30 third graders. Indexes of internal consistency were computed using the Kuder-Richardson formula 20. The results were .86, .88, and .85 for the pretest, posttest, and follow-up test, respectively. To demonstrate equivalency of the three tests, mean scores were calculated. The scores on the pretest
averaged 80% correct, whereas the posttest and follow-up scores averaged 81% and 83% correct. Data for all three tests are presented in Table 2.
Trainer Instruction The seven trainers who participated in the study were students who were majoring in special education and who had some classroom experience. Five trainers instructed more than one group of students; however, each trainer taught only one kind oflesson. Trainers attended five l-hour sessions over a 2-week period before the beginning of the study. The assignment of trainers to groups was made on the basis of the times that they were available for instruction. The training for each treatment occurred in the same manner, but was conducted separately for the trainers in each treatment. Using role-playing techniques, the experimenter demonstrated the lessons for each of the treatment conditions. Following the demonstrations, each person was given a chance to act out the part of the instructor while others acted as observers. Trainers were required to score 80% or above on three consecutive demonstrations before they could participate in the study.
Procedural Reliability To determine the fidelity with which the lessons were presented, each trainer was observed twice during the course of the study. Observers, who also were undergraduate special education majors, were trained to complete an observational 515
Exceptional Children
Downloaded from ecx.sagepub.com at Bobst Library, New York University on January 26, 2016
protocol with which trainers' presentations could be evaluated. The presentations of trainers in the strategy-plus-sequence and strategy-only conditions (i.e., trainers who taught the big number concept) were judged on the presence or absence of six elements of their scripted lessons; trainers in the sequence-only condition were judged on seven elements. The criterion for reliable administration of the lessons was at least 80% "yes" responses. The reliability scores averaged 89% and ranged from 83% to 97%. No trainer ever fell below the 80% criterion and required reteaching. Treatment Similarities Although the students in the three treatment groups were taught with different instructional methods, the treatments had several features in common. In all training conditions, groups of three to five students were engaged in instruction for 14 30-min sessions. Instruction occurred Monday through Friday for 3 weeks. Another similarity was the use of detailed, scripted lesson plans. All groups received the same number and identical sets of word problems in both the boardwork and seatwork phases of the lessons. During seatwork, trainers were available to answer any questions students had; and they reinforced students for accurate work and appropriate social behavior. Strategy Only and Strategy Plus Sequence Fourteen 30-min lessons were developed for both treatments according to the recommendations of Silbert et al. (1981, chapter 12). Students responded in unison to questions. Hand signals were used to control responding, as well as to ensure that all students in the group were participating. During the first 2 days of training, the groups were given instruction on the fact-family concept. This instruction was presented in three ways. First, the rule about what to do when the big number was given was taught. For example, the students were told that "When the big number is given, subtract." After telling students the rule, the trainer demonstrated its application, writing a diagram on the board in which the big number was given. Next, the trainer introduced the rule about what to do when the big number was not given. This application was also diagrammed on the board. After approximately 15 min of boardwork practice, students were given seatwork practice sheets.
On Day 3, students were taught to apply the big number concept to solving word problems. Although both the strategy-plus-sequence and the strategy-only groups received the same strategy instruction, the presentation of practice problems differed. The strategy-plus-sequence groups received all of the same kind of practice problems. The first three lessons involved simple action problems, followed by three lessons of classification problems, three lessons of complex action problems, and three lessons of comparison problems. The strategy-only group received a balanced combination of practice problems that included all four types each day. Sequence Only The 14 30-min lessons that comprised the sequence-only group were adopted from Mathematics Today (1985), the basal math series used in the students' schools. The first 2 lessons involved practice in the computation of addition and subtraction problems. On Day 3, the instruction on word problems began. The boardwork portion of the lessons involved the use of a questioning technique that prompted students to see information in the problem. A typical question might be, "What numbers are given in the problem?" or "What are we supposed to find?" As the children answered the questions, the trainer wrote their answers on the board. Once boardwork was completed, students were assigned seatwork problems. The students in this group practiced both boardwork and seatwork problems that had been grouped according to problem type in the same manner as the strategy-plus-sequence group. That is, the first three lessons involved simple action problems, followed by 3 days of classification problems, 3 days of complex action problems, and 3 days of comparison problems. RESULTS Data were analyzed to determine if the independent variable of type of instruction (strategy plus sequence, strategy only, sequence only) affected the dependent variable of number of problems solved correctly. A problem was counted correct if the correct algorithm was used in computing the answer. Analysis of covariance (ANCOVA) was used, with the pretest serving as the covariate. Separate ANCOVAs were conducted on the posttest and the follow-up tests. Students rather
516
May 1991
Downloaded from ecx.sagepub.com at Bobst Library, New York University on January 26, 2016
than group served as the unit in these analyses because the use of trained experimenters, scripted lessons, and prescribed examples minimized variation due to teachers. Table 2 contains the obtained and adjusted means and standard deviations for the three experimental groups. The obtained means were adjusted statistically to take into account the initial differences among the pretest means. The adjusted means represent the best possible estimates as to what the groups would have scored on the posttest and follOW-Up measures if the three groups had started with identical pretest means. On the analysis of the posttest data, a significant effect for treatment was found, F(2, 58) = 4.69,p = .013. To determine specifically how the treatments differed, post hoc analyses were conducted using the adjusted means for each treatment, according to the procedure described by Pedhazur (1982) for ANCOVA. Although students who received instruction in the strategy-plus-sequence group scored higher than did students in both the strategy-only and sequence-only groups, the post hoc analysis indicated that the strategy-plus-sequence group differed significantly only from the sequenceonly group, F(1, 57) = 7.49,p < .05. Further, students in the strategy-only group also scored significantly higher on the average than did students in the sequence-only groups, F(1, 57) = 4.33,p < .05. The ANCOVA for the follow-up test also yielded a significant effect for treatment, F(2, 58) = l4.71,p= .0001. The post hoc analysis revealed that the adjusted mean score of the strategy-plussequence group was significantly higher than the adjusted mean scores of the sequence-only and strategy-only group. Furthermore, the strategyonly group scored significantly higher than did the sequence-only group. An analysis was undertaken using unadjusted means to describe the performance of the LD students, taken as a whole, relative to the third-grade normative sample. These results are presented in Figure 1. On the pretest, the LD students solved 5.41 fewer problems than did the third graders; on the posttest, this difference decreased to 3.96. Thus, on the average, the LD students in this study approached, but never reached, the level of performance of the third graders despite the fact that all had participated in 14 lessons on solving word problems. On the other hand, the pretest to posttest improvement for the LD students (M = 1.61) was statistically significant, 1(61) = 3.77, p
< .001, despite the fact that the effects of the three treatments were averaged. DISCUSSION The analysis of the data indicated significant differences among the groups on both the posttest and the follow-up test. Students in the strategyplus-sequence group scored significantly higher on both measures than did students in the sequence-only group. Students in the strategy-only group also scored significantly higher on both the posttest and the follow-up test than did students in the sequence-only group. However, on the posttest, the difference between the strategyplus-sequence and strategy-only groups was not significant; on the follow-up test, the strategyplus-sequence group scored significantly higher. One explanation for this difference may be the continued use of the big number concept by a classroom aide, unbeknownst to the experimenters. The aide picked up on the strategy and continued its use with a strategy-plus-sequence group. Thus to conclude that strategy plus sequence produced a more enduring effect than strategy only would be an arguable point. Nonetheless, the data indicated the superiority of a direct-instruction approach for teaching students with learning disabilities to solve addition and subtraction word problems, regardless of whether that direct instruction approach incorporated sequenced examples. Our results were consistent with those of Darch et al. (1984), but not with Jones et al. (1985). In the present study, as well as that of the Darch et al., the direct-instruction approach led to superior performance when compared with a basal approach. It should be pointed out that both studies used the big number concept for strategy teaching and that the basal approach in the Darch et al. study was similar to that used in the present study. In this study, we sought to refine the findings of Jones et al. (1985), who also examined the effects of sequencing problems according to type versus teaching all problem types concurrently. In the present study, however, the design permitted an assessment of the effects of strategy teaching and sequencing in isolation. The comparison of the strategy-plus-sequence and strategy-only groups represented an exact replication of the Jones et al. (1985) study; however, Jones et al. reported that students who received strategy-plus-sequence instruction scored higher than did those in the strategy-only group.
Exceptional Children
517
Downloaded from ecx.sagepub.com at Bobst Library, New York University on January 26, 2016
FIGUREl Pretest and Posttest Performance of the Two Samples: Students With Learning Disabilities (LD) and Third Graders Without Disabilities
25.,----------------------,
3rd Graders
20
LD
III
c
co
