163
A Formula-based Classification of Learning Disabled Children: An Examination of the Issues
Louis C. Danielson, PhD, and Jane N. Bauer, MA
With the passage of the Education for All Handicapped Children Act, the Bureau of Education of the Handicapped (BEH) was charged with the task of establishing regulations for enactment of the new law. The Federal Register (December 29,1977) reporting the final LD regulations notes that an analysis was made of all comments and testimony regarding the proposed regulations of November 1976. This report, prepared by two BEH scientists, first analyzes the numerous comments, and second describes some BEH in-house research which, along with the comments, prompted the bureau to drop the IQachievement formula in the final regulations. —G.M.S.
W
ith the publication on December 29, 1977, of final learning disabilities (LD) regulations, states and local school districts are left with the task of implementing the procedure for the identification of LD children.* While the application of the proposed LD regulations had been based on a formula, the overwhelming opposition to the use of the formula led to the elimination of both the formula and the specification of a specific severity level for eligibility. During a 120-day review period, the Bureau of Education for the
Handicapped (BEH) undertook some field testing of the formula with data supplied by states and local school districts and also analyzed solicited comments on the adequacy of the formula-based procedure. Some of its findings should be very pertinent as states and local education agencies move to implement the final LD regulations. It is felt that some of the problems and issues related to the use of the federally proposed formula will be typical of other procedures as well. Therefore, this article will outline some generic issues and problems related to the development of operational procedures for identifying LD children.
'The opinions expressed herein do not necessarily reflect the position or policy of the U.S. Office of Education, and no official endorsement by the Office of Education should be inferred.
The final regulations (U.S. Office of Education 1977b) provide that a child has a learning disability under specific conditions:
50
Journal of Learning Disabilities Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
164 (1) The child does not achieve commensurate with his or her age and ability levels in one or more of the areas listed below, when provided with learning experience appropriate for the child's age and ability levels; and (2) The team finds that a child has a severe d i s c r e p a n c y b e t w e e n a c h i e v e m e n t and intellectual ability in one or more of the following areas: (a) oral expression, (b) listening comprehension, (c) written expression, (d) basic reading skill, (e) reading comprehension, (f) mathematics calculations, or, (g) mathematics reasoning. (3) The team may not identify a child as having a specific learning disability if the severe discrepancy between ability and achievement is primarily the result of a visual, hearing, or motor handicap, emotional disturbance, or environmental, cultural, or economic disadvantage.
THE PROPOSED FORMULA The proposed regulation (U.S. Office of Education 1976) had provided the same basic procedure for determining the existence of a specific learning disability; however, they had relied on the use of a formula. Where the final regulations leave the "severe discrepancy" undefined, the proposed regulations defined it as achievement at or below 50$ of the expected achievement in at least one of the several achievement areas, after accounting for age and previous educational experience. In the proposed regulations, the formula for determining the severe discrepancy level (SDL) was: CA / ^ + 0 . 1 7 \ - 2 . 5 = SDL, \300 / where SDL is expressed as a grade equivalent (GE) and CA represents chronological age. As given above, Formula 1 is a transformation of the familiar Harris (1970) formula for estimating a reading age expectancy: Expectancy Age =
2MA
+CA 3
where M A = mental age. This can be converted to a grade expectancy as follows: 2MA+CA _s G E = 3 The SDL which is 50$ of expectancy is then obtained by Formula 2: SDL = ( 2 M y C A ) - 5 » . 5 . This formula is more intuitively appealing than Formula 1 in that it reflects the weighting of MA and CA with conversion to grade equivalents and the subsequent multiplication by .5 to get the 50$ of expected functioning. However, due to objections concerning the use of mental age, MA was replaced by CA -^ in the formula. In addition to this substitution, terms were rearranged to yield Formula 1. The proposed formula was intended to identify children having extreme problems with learning as demonstrated by their achievement, and to provide a common standard for counting children with specific learning disabilities.
ISSUES RELATED Many comments received by BEH during the 120-day period dealt with generic issues concerning the use of a formula. Some comments relating to specific aspects of the proposed formula were interesting because they dealt with potential areas of concern for alternate procedures, i.e., use of IQ scores, use of grade equivalent, etc.
Intelligence Quotient The student's IQ was included in the proposed formula as a measure of ability and was used along with CA to calculate expected achievement. Comments on the inclusion of IQ in the formula raised the issues of validity and reliability and of its appropriateness with children who are learning disabled, of low socioeconomic status (SES), or of preschool age. Since the final regulations seem to require the assessment of intellectual ability, the use of IQs will continue to be a concern of many people.
Volume 11, Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
51
165 Validity of IQ. A number of commentors questioned the use of IQ on the basis of its validity either as a measure of ability or as a measure of potential. Performance on an IQ test may be in part attributable to innate ability; however, since the test depends upon responses which are obviously learned, other characteristics such as test-taking abilities, fatigue, anxiety level, native language, and prior experience with similar tasks affect performance. These characteristics are largely a function of the individual's environment. Therefore, the degree to which an IQ test score can be taken to indicate intelligence may be indeterminate. Another expressed concern is the fact that an IQ score is an aggregate of many discrete abilities and that IQ tests assess different abilities. A child's assessed ability would, therefore, vary depending on the IQ test used. The validity of an IQ test as an estimator of achievement would in turn vary, based on the relationship between those abilities and the achievement in the academic areas of interest. The proposed regulations were also faulted for not attending to the fact that there will be an IQ score error of measurement. The reliability of IQ tests was questioned in general and specifically for preschool children. Appropriateness of IQ. In addition to these general criticisms, there were issues related to the use of IQ scores with specific groups of children: learning disabled, low SES, and high, or low IQ. A substantial proportion of commentors expressed the opinion that being learning disabled would preclude the valid assessment of the child's IQ. Thus, the formula would less likely label this child appropriately as LD. Section 121a.532 of the PL 94-142 regulations addresses this issue (U.S. Office of Education 1977a). The regulations require that a test which purports to measure aptitude accurately reflect the child's aptitude and not his impaired sensory, manual, or speaking skills. Although the regulations seem to prohibit the use of an IQ test which would be affected by a child's learning disability, assessing the child's IQ remains problematic. 52
Many commentors felt the use of IQs in the formula would cause discrimination on the basis of SES. Some felt the procedure would result in the inclusion of too many low SES students in LD programs while others felt it would include too few. A few people felt that children of low SES would do poorly on IQ tests and therefore would be less likely to qualify as LD. One commentor expressed the opinion that since IQ and achievement test performance would be equally affected by SES, the formula could have worked well. The LD regulations state that a child cannot be identified as LD if that severe discrepancy is primarily the result of environmental, cultural, or economic disadvantage. However, our ability to state that a severe discrepancy exists will undoubtedly be much greater than our ability to infer whether it was caused by SES. A few commentors suggested that the formula might identify a greater proportion of children with low IQs than with high IQs. In addition, it is quite possible that there would be fewer high IQ children referred for evaluation, so a formula which is less likely to identify these children combined with fewer children being referred could result in very few above average IQ children being identified as LD. Summary. The major concerns about using IQ in the proposed procedure can be reduced to six points: (1) IQ is not an index of "ability" or "potential." (2) Since intelligence as operationally defined in most tests is an aggregate of many discrete abilities, and tests vary in their composition of these abilities, estimates of ability may vary from test to test. (3) Intelligence as a predictor of achievement will vary from one area to another. (4) The very fact that a child is learning disabled may preclude the valid measurement of intelligence. (5) The fact that a classification procedure relies upon IQ invites cultural discrimination. (6) Incidence figures produced by the formula would not be uniform across IQ levels. Journal of Learning Disabilities
Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
166
Chronological Age Comments related to chronological age primarily concerned the interaction of prevalence with age and the fact that the formula did not account for years in school. Several respondents felt the BEH formula discriminated against older students while others believed the formula discriminated against younger students. It is unlikely that both contentions could be correct since the formula should yield a linear relationship between the SDL and age. Data related to this issue are presented in a subsequent section. Another issue cited, somewhat related to age, contends that the formula would not account for previous educational experience. The formula contained CA, but this obviously translates imperfecdy to years in school by subtracting five since children may vary at least one year in the age they begin school.
Severity Level The preamble to the proposed regulations stated that a severe discrepancy exists when achievement falls at or below 50$ of the child's expected achievement — a figure representing "the level at which a child's educational performance is clearly impaired." The 50$ discrepancy had been incorporated into the formula so that the formula yielded half of expected achievement. This 50$ value would account most directly for the proportion of children who were identified as LD. Many respondents stated that the 50$ figure was arbitrary and were opposed to it. Sulzbacher and Kenowitz (1977) raised two questions concerning the 50$ discrepancy: Is a 50$ discrepancy in math as disabling as a 50$ discrepancy in spelling? Is a 50$ discrepancy for a third grader as deserving of remediation as a 50$ discrepancy in a tenth-grade student? (p.68) LD prevalence would be larger if a value higher than the 50$ level were used. The estimates of prevalence that respondents felt the formula would produce ranged from 1 to 30$. A number of people suggested keeping the 2$ limit
on the number of students a state could count as LD because of the possible inadequacies of the formula. Most of those responding on the basis of prevalence felt the formula would be too restrictive; however, many of these people felt the formula would produce a prevalence figure very near 2$. A number of these individuals suggested that a desirable prevalence figure would be 3 or 4$.
Measurement of Achievement Both the proposed and final regulations specified in part that "a team may determine that a child has a specific learning disability if... the team finds that a child has a severe discrepancy between achievement and intellectual ability." To use the proposed formula, one had to express achievement as a grade equivalent. In implementing the final regulations, some attention should be given to the advantages and disadvantages of various scales. A grade equivalent is a comparatively crude scale and lacks many desirable psychometric properties. For example, a score of 4.2 in reading for a sixthgrade student indicates only very generally the student's standing relative to other students or his mastery of reading. However, grade equivalents are firmly entrenched in the public schools and very familiar to school personnel. Some commentors felt that the focus on standardized norm-referenced testing was undesirable. These respondents supported the use of criterion-referenced instruments and felt that the regulations would discourage the use of these instruments. Further, a number of commentors stated that normed tests are generally unavailable for some of the achievement areas specified in the regulations, and that where tests are available, they are inappropriate for certain groups, e.g., young children and minority children.
Utilization of the Proposed Procedure Both the proposed and the final regulations specify that a child must display a severe
Volume 11, Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
53
167 discrepancy between achievement and ability. The procedure specified by the proposed regulations required that the SDL as determined by the formula be compared to achievement test scores. A child would have been learning disabled according to the formula if achievement ^ SDL, or if achievement - SDL < 0. It is apparent that this decision would have been based on a discrepancy score, which can be quite unreliable (e.g., Cronbach & Furby 1971). The final regulations imply that a discrepancy score would be involved even though it has not been operationalized. One individual responding to the proposed regulations suggested that no more than 25% of those identified as learning disabled by the formula would be so identified in an independent replication of the procedure. In addition, a child would have been required to exhibit the severe discrepancy in only one of the eight achievement areas to qualify as learning disabled. Therefore, the probability of a child exhibiting a severe discrepancy in one of these eight areas due to chance alone could have been substantial. The final regulations now have seven achievement areas. As the proposed regulations were presented, the formula would define a SDL which would be a constant for a child of a given age and IQ and which would then be the same regardless of the deficient achievement area. Several commentors questioned whether the relationship between age, IQ, and achievement is such that the same formula could be used to determine the severe discrepancy in each of the achievement areas. A number of respondents pointed out that there were problems in using the formula at early childhood levels. The proposed regulations pointed out that the formula could result in negative values at these ages although this in itself is not troublesome since they can be translated to grade equivalents, for example, -0.1 equals prekindergarten 0.9. The formula may have had more serious problems at the preschool level; for example, the formula yields a value of -.5 for a four-year-old child with an IQ of 100. If one converts that to an age equivalent by adding 54
5 (the formula originally converted an age expectancy to a grade expectancy by subtracting 5), the severe discrepancy for this child is an age equivalent of 4.5. The four-year-old child would have been LD according to the formula if the child had achievement equal to or less than an age equivalent of 4.5. The formula seems to yield unreasonable values for children younger than five years old since the severe discrepancy is greater than the level at which one would expect the average child to be achieving. This problem becomes more pronounced the lower the age. Some of the respondents referred to the difficulties in measuring achievement at preschool levels. There are obvious problems with reliability and instrumentation at younger ages as well as with the issue of what constitutes a learning disability at the preschool level. For example, the question arises as to what reading achievement is at an early childhood level. A number of commentors expressed the opinion that the regulations could serve to mediate against the early identification and treatment of LD children. A clear concern of a number of respondents involved the problem of distinguishing between measured intelligence as an indication of the child's innate characteristics and achievement as a learned characteristic. This dilemma may be most clear at a preschool level where tests of prereading skills involve the same kinds of tasks as intelligence tests. It had been suggested that the procedure had no established validity; that is, there had been no demonstration of its capability of identifying those children who are in fact LD. Unfortunately, this validation would have been impossible since agreement on which children were or were not LD would have been virtually impossible to obtain. A subsequent section examines the degree to which the proposed formula identified those children who had previously been determined to be LD. The BEH formula could have varied from existing procedures in the restrictiveness of its results; i.e., it may have identified children with Journal of Learning Disabilities
Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
168 TABLE I. Projected incidence figures resulting from the application of the formula to actual data. A
B
Data LD sample Present base size prevalence 1 2 3 4 5 6 7 8 9 10 11 12 13 14
246 101 771 2,428 943 130 180 577 414 82 80 30 63 424
.043 .028 .034
.020
.014 .029 .059 .046 .029
C
D
E
|
F
LD qualifying Proportion of non- Total proportion by form ula Projected LD with IQ ^ 80 qualifying N % prevalence* qualifying by formula (D + E) 93 38 50 50 559 63 1,430 59 439 46 74 57 97 54 363 63 177 43 65 79 59 74 26 87 60 95 279 66
.016 .014 .021
.011
.006 .023 .044 .039 .028
.102 .077 .119 .094
.125 .121 .158 .122 I
*These figures were obtained t •>y multiplying the present prevalence figures by the proportion of the LD students who qualified by the formula. Prevalence refers to the number of LD children divided by the school-age population.
more or less severe handicaps while their characteristics remained the same. Alternatively, the formula may have varied from previous procedures in the definitions it gives, in that its implementation may have constituted a change in definition and, therefore, a change in the characteristics of the children identified. A planned research effort would have been n e c e s s a r y to d e a l w i t h t h e s e i s s u e s comprehensively. Some commentors argued that a change in definition was implicit since the process disorder aspect of identification had not been attended to. Finally, a number of commentors responded that the implementation of the formula would have been difficult since testing would be timeconsuming and teachers were not adequately prepared to use the formula. Another question related to the need for criteria for determining when a child no longer required special services — whether an LD child was ineligible once his achievement level was greater than the SDL defined by the formula.
EMPIRICAL ANALYSES OF FORMULA To examine the applicability of the proposed formula, we obtained information from 14 existing data bases. Since data collection was post hoc and d e p e n d e n t on voluntary cooperation, there are several limitations of the study. Only two data bases (Data Base 4 and 9) included an urban area; a wide range of tests was used to measure achievement; not all children were administered the same number of tests; and in some cases, the population from which the sample was drawn was not known. However, these variations reflect to some degree the varied national procedures. Because of the procedures used to obtain the data bases, these data could not be taken to constitute a sample from any identifiable population. Given the variety and quantity of data, we did feel, however, that if the formula performed poorly on these data, it would not be tenable. On the other hand, if it performed
Volume 11, Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
55
\
169 TABLE II. Percentage of children by IQ level who would be classified asLD by the \ | proposed formula. Data bases for children currently classified as LD IQ 80-90 N
%
91-110 N
%
111 + N
%
1
2
18 44
21 53
75* 68
46 35
24 56
280** 71
22 40
1 25
204f 75
3
Data bases for students with IQ > 80
9
10
11
12
13
10
11
12
13
47 41
32 64
21 75
10 83
29 94
104 40
42 34
34 39
152 39
101 41
25 81
30 71
12 86
30 97
193 24
97 16
54 19
20 20
25 58
8 100
8 80
4 100
1 100
62 9
24 7
16 9
10 5
I
* I Q ^ 84 **IQ = 85-100 f l Q > 101
adequately, we could not presume that it would work as well on other samples. The data are discussed in the following sections according to the question they address.
What Size Prevalence Figures Would Result from the Formula? Many commentors expressed concern over the possible number of children to be identified as LD by the formula because of this information's great fiscal importance, in addition to its effect upon the educational programming and service received by many children. We were interested in ascertaining how the formula might affect the absolute number of children counted and also in anticipating how it might affect whether individual LD children continue to be counted. Prevalence figures (proportion served) for these data bases vary from a low of .014 to a high of .059 (Table I). When the formula was applied to these LD children, between 38$ and 95$ of the children still qualified as LD. For Data Base 4, 56
59$ of the LD children qualified on the basis of the complete agreement of the evaluation team (an alternate method for children to qualify as LD). We calculated a projected prevalence figure by multiplying the proportion of the LD students qualifying under the formula by the present prevalence. This figure represents a minimum prevalence figure because it does not include those children who are not currently identified as LD but would be so identified by the formula. Data in column E reflect the proportion of children who exhibit the "severe discrepancy" but who are not in an LD program. Column F gives the total proportion of children in the data bases that were identified as LD by the formula. Representing an upper limit on the prevalence figure, these figures suggested that if all children in a district with an IQ over 80 were tested in all or nearly all of the eight achievement areas, a prevalence figure as large as 16$ might not be unreasonable. Of course, the prevalence figure Journal of Learning Disabilities
Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
170 I
TABLE
I
as LD by the proposed
III
Percentage of children
.% 9-12 N
%
13 and above N
%
age who would
Data bases for children currently classified as LD
Chronological age
8 and below N
by chronological
be
classified
formula. Data bases for students with IQ > 80
1
2
9
10
11
12
13
68 65
10 29
13 26
17b 85
5b 83
10b 83
17b 100
80 62
14 39
57 30
25° 64
50c 71
14c 88
33c 92
5 45
27a 90
107 61
64 100
d
14 100
d
a
d
2 100
14
10
11
12
13
33e
94 29
15 15
35 34
66 34
57f
190 19
102 14
52 16
138 21
85,77g
75 19
46 19
17 14
64 27
d
10 100
"Grade 1-3 f Grade 4-6 8 Grades 7-9, 10-12
13 and above Below 8 "8-12 d Over 12 b
obtained would probably be dependent on the achievement tests used. Data presented later lead one to suspect this might be the case.
To What Extent Do Estimated Prevalence Figures Vary across IQ Levels? Some commentors felt the BEH formula would not yield uniform prevalence figures across IQ levels. It should be noted that some individuals felt that prevalence figures should not be equal across IQ levels. Data presented in Table II address this issue. Examination of the data on the percentage of LD children qualifying according to the formula across IQ levels suggests that there is no clear variation across IQ levels. As we examined the IQ distribution for the LD children in these data bases it became apparent that the distributions were generally irregular and varied considerably from one data base to another. Since the original distributions were unusual, it is probably unwise to draw any inferences
concerning uniformity of incidence across IQ levels. The form of the distributions may in part be attributable to nonuniform referral and classification procedures and to policies which prohibit children with IQs less than some absolute value from being classified as LD. We felt it might be more appropriate to address the question above by applying the formula to all school-age children in several districts. We first excluded children with IQs less than 80 and then applied the formula to the remaining children. The data show that children of 80 to 90 IQ are much more likely to be determined to be LD by the formula than children with higher IQs (Table II).
To What Extent D o Estimated Prevalence Figures Vary across Age Levels? We found that children were not uniformly distributed across age levels in our LD data bases so again we applied the formula to the entire
Volume 11, Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
57
TABLE IV. Agreement between Wide Range Achievement Test (WRAT) and Peabody Individual Achievement Test (PIAT) subtests in classifying children as LD. Percentage of LP children qualifying by test Test
Reading
Arithmetic
Spelling
PIAT only WRAT only Both PIAT & WRAT Neither N
4.5 7.5 12.0 76.0 67
4.6 22.7 1.5 71.2 66
4.5 15.9 27.3 52.3 44
school-age population with IWs greater than 80 for several districts. Based on the data (TableIII), we feel that children under eight years old are somewhat more likely to be identified as LD by the formula than are older children.
the norms for the math tests differ. There was considerably more overlap between the PIAT and WRAT in the proportion of children found to be LD for spelling and reading (Table V).
The percentage of LD children aged 13 and above who were identified by the formula is somewhat larger than the percentage of those LD children aged 9 through 12. We suspect that this may have resulted because these districts are serving fewer adolescent children who represent the more severely handicapped children.
Does the Proportion of Qualifying Children Vary according to Achievement Area?
To What Extent Does the Effect of the Formula Depend on the Achievement Test Used? Since there were only 66 children who had taken more than one achievement test, the sample for this question was smaller than the sample for the other questions. Data for those 66 children for whom scores on both the Wide Range Achievement Test (WRAT) and the Peabody Individual Achievement Test (PIAT) were available are presented in Table IV. On the basis of the WRAT math score 16 of these children qualified as LD while only four children qualified on the basis of their PIAT math score. Just one child qualified on the basis of both scores. The correlation of the math tests was high, which indicates that these children scored in much the same relative position on each set of tests. These results lead one to suspect that 58
A number of respondents suggested that the formula might qualify relatively larger numbers of children in some achievement areas such as spelling and fewer children in other areas such as math. One is led to believe that this might be the case based on the data presented in Table VI. Among the LD children of Data Base 9, a larger number qualify on the basis of their spelling test on both the WRAT and PIAT. Comparatively fewer children qualify on the basis of their math scores on both the PIAT and the WRAT. On the Stanford Achievement Test, larger numbers qualified on reading recognition and spelling than on both math concepts and math computation. Using standard scores in place of grade equivalents would probably correct this as the outcome is likely related to differences in the variance for these subtests.
DISCUSSION In an attempt to integrate the issues generated with the results of the empirical analyses, the discussion is organized around three questions to Journal of Learning Disabilities
Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
172
TABLE V. Correlations between Peabody Individual Achievement Test and Wide Range Achievement Test subtests from Data Base 9.
WRAT subtests Reading Spelling Math
PIAT subtests Reading Recognition Spelling .91 .63 .86 .79 .61 .50
Mathematics .62 .61 .74
NOTE: Sample sizes ranged from 44 to 67.
determine whether a formula-based procedure should be adopted. (1) Is the formula effective? (2) What would be the impact of the formula? (3) Can the procedure be implemented?
Effectiveness of the Formula For the purpose of this discussion we define effectiveness as the extent to which the procedure meets its goal of identifying those LD children with the most severe problem. Implicit in this goal are two criteria for evaluating the effectiveness of the proposed procedure. (1) Does the procedure reflect what might be taken as the appropriate criterion? (2) Does the procedure identify those children who are the most severely handicapped given that criterion? The procedure identifies as learning disabled those , children at the lower end of the achievement-minus-SDL curve. Because the children who are lowest on this criterion are identified as LD, it obviously identifies the most severely handicapped only if one agrees that the proposed procedure reflects the appropriate criterion. If one accepts the proposition that the learning disabled child is one who exhibits a severe discrepancy b e t w e e n e x p e c t e d achievement and actual achievement, with expected achievement being a function of age and IQ, then the procedure would appear to reflect an appropriate criterion. In general, few
commentors expressed dissatisfaction with the criterion of discrepant achievement. However, some expressed the opinion that the procedure should reflect process disorders. Other commentors questioned the reliance of the procedure upon IQ. Many of these commentors felt that an IQ test is not a valid measure of the ability of LD children. Thus, one might agree conceptually with the notion of abilityachievement discrepancy as a criterion but fault the way it is implemented. If one could identify a group of children who are in fact LD, then the procedure could be directly validated and the appropriateness of the criterion would not be significant; however, the lack of agreement over which children are and are not LD precludes this analysis. In the absence of a generally accepted standard for determining the appropriateness of the proposed procedure, one indication of its validity is the extent to which currently identified LD children are identified by the proposed formula. Across all data bases in our analyses, 58% of the children currently classified as LD were classified LD by the formula. However, the variation between data bases was large, ranging from 38 to 95% of the LD children who would qualify as LD by the formula. Also, the formula should not identify as LD those children who are not LD. Among those children not currently classified as LD, 8 to 12% were classified as LD by the formula.
Volume 11, Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
59
173 TABLE VI. Percentage of students who qualify as LD on specific subtests of the Wide Range Achievement Test, the Peabody Individual Achievement Test, and the Stanford Achievement Test when the formula is applied. Test N LD students — Data Base 9
%
WRAT Reading Spelling Mathematics
82 118 63
23 36 18
PIAT Reading Recognition Spelling Mathematics Reading Comprehension
16 32 9 13
14 30 8
12
Entire student population with IQ ^ 80 — Data Base 11 SAT Reading A 38 9 Reading B 57 14 Reading Comprehension 95 5 Work Study Skills 78 6 Spelling 136 9 Language 104 8 Math concepts 69 4 Math Computation 75 4
technical issues of reliability and scale, which are discussed more fully in a subsequent section. Third, the proposed procedure might differ in substance from procedures currently used. If so, then validity is a major issue. If children labeled LD by one procedure are not so labeled by another procedure, then one or both of the procedures must be inappropriate. Fairness of procedure. One might assume that an appropriate procedure should be fair, according to age and culture. One way to determine fairness is to look at projected prevalence figures across age and IQ. We do not suggest that prevalence should necessarily be equal across IQ, age, and culture; however, in the absence of reasons why it should not be equal, this may be one way of examining fairness. It was found that children with lower IQs seem to be more likely to be identified as LD by the proposed formula than children with higher IQs.
There are several reasons the proposed procedure may not identify all LD children currently served. First, the proposed discrepancy level of 50% may be more severe than many LE As or SEAs are using. Therefore, the procedure identifies only the more severely handicapped of the children currently classified as LD.
The potential impact of using a formula which identified greater proportions of children with lower than with higher IQs might be to cause large numbers of the population identified as LD to be slow learners. There also appears to be a slightly greater probability that younger children would be determined to be LD by the formula than would older children. But, in light of the fact that many commentors were concerned about early identification of the learning disabled, this may not be a troublesome finding. We were unable to address the cultural fairness of the procedure with the data that we obtained. A number of commentors suspected that the procedure would be unfair to some minorities because of its use of IQ.
Second, if the test data used to evaluate the formula were not the same test data used to place the children, then variations between test and retest might lead to different decisions. Since these children are selected on the basis of extreme discrepancies, statistical regression might be anticipated to affect many decisions on a test-re test basis. The problem of test-retest variation in the outcome of applying the proposed procedure is an aspect of the more
Consistency of procedure. An additional measure of effectiveness is the procedure's reliability. If the decisions based on the proposed procedure depended to a large degree upon chance, one could not have seriously considered the procedure viable. The proposed formula involved the use of IQ tests and achievement tests, which can be very reliable. Many school psychologists rely on individual intelligence tests such as the Stanford-Binet or
60
Journal of Learning Disabilities Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
174 the WISC, which are highly reliable. In fact, most of the LD children in our data bases, for whom data were available, were given the StanfordBinet, the WISC, or in some cases, the Slosson. Most frequently the achievement tests used were the WRAT and the PIAT. Reliability for these tests varies by age and by subtest, but as an example, the WISC Full Scale Score reliability is generally above .9 and the median reliability of a PIAT score is .78 (Dunn & Markwardt 1970). The procedure, however, involves the use of a discrepancy score, the reliability of which will be reduced when the tests are positively correlated. As an illustration, given reliability values of .8 and .9 for achievement and IQ, respectively, and a .7 correlation between them, the reliability of the difference would be about .4. If one considers the standard error of measurement in making decisions about individual children, the reliability of the procedure is such that, to be very confident that a child is in fact LD based on his demonstrated discrepancy, the child would have to exhibit an achievement score considerably lower than the SDL. Thus, if one wished to minimize the probability of erroneously classifying children as LD, many children who are LD would not be so identified on the basis of their observed scores. On the other hand, if one wishes to maximize the probability of identifying all children who are in fact LD, one would be likely to identify a substantial number of children who are not LD. Since the procedure identified children who score in the extreme end of the lower tail of the discrepancy curve, the problem of statistical regression must also be considered. This phenomenon, a function of the reliability of the procedure and the distance of the scores from the mean, could result in only a small percentage of those labeled as LD on initial testing, labeled LD at a later testing. One commentor suggested that only about 25% of those children labeled LD on one occasion would again be so labeled in an independent replication of the procedure (Cronbach personal communication, 1977). As was indicated by the data, there may be
some variation in the decision the formula would yield, depending on the particular test used. Although this variation is troublesome, we may be demanding too much from tests if we expect them to be sensitive in the extreme tails. These differences could as well be artifacts of the procedure used to create the grade equivalent scale for the tests. In any case, test variation in terms of the decision they produce would seem to be a useful concern of further research. Another component of the procedure which may affect decision consistency is the use of the grade equivalent. Although the grade equivalent may be a familiar scale to most educators, its use may be a limitation of the proposed formula. As Angoff (1971) very aptly put it: The principal claim that can be made for the grade and age equivalents is that they have a simplicity and directness of meaning in terms of the test user's everyday experience that are not shared by other scales. However, the difficulties and confusions that are attendant on the use of these equivalents would indicate that their simplicity is far more apparent than real and that the truly simple scales may well be those for which there has been no attempt to capitalize on the use of direct meaning. Moreover, while it is possible that direct meaning may be a highly desirable feature in a system of derived scores, the trouble is, as has frequently been pointed out, that users read into such scores more or different meaning than they actually possess, (p. 525) Although not directly addressed in the comments, this problem could be resolved in part by substitution of some other scale, such as standard scores, for the grade equivalent. An extensive review of the methodological limitations of grade equivalents is provided in Angoff and the reader is referred to this source for further elaboration of this issue.
IMPACT OF THE PROPOSED PROCEDURES We define impact as a measure of the consequences of the proposed procedure which are not directly related to the explicit goals of the procedure. We anticipate that the number of LD children that would continue to be labeled LD by
Volume Uy Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
61
175 the proposed procedure would vary substantially from place to place and would be very much dependent on the particular evaluation and placement system in place in an LEA. It seems safe to say that the proposed procedure seems to be more restrictive than many LEAs are currently using. However, variations in the way the procedure might be implemented (e.g., tests used, number of areas tested, proportion of children referred for testing, whether the standard error of measurement is considered) could produce large variation in prevalence. Using this procedure should produce more uniformity in the population of children referred to as learning disabled. Again, the manner of implementing a procedure will affect the consistency of the outcome.
IMPLEMENTATION OF A FORMULA-BASED PROCEDURE The final issue that we address is whether a formula-based procedure can be implemented. Based on data supplied to us, it seems reasonable to conclude that LEAs are already collecting the test information that would be required to use the formula procedure discussed above. This formula is easy to apply; however, there may be some resistance to using this procedure since there were many comments received which were less than favorable concerning the formula.
SUMMARY In sum, the results of the empirical analyses of the formula presented in the proposed regulations suggest that the procedure may have utility. There appear to be some problems with the values the formula produces for preschool children, and the formula may be judged by some to yield prevalence figures that are unacceptably large for slow learners. The whole procedure is plagued by numerous needs for value decisions. For 62
example, if many children are identified on the basis of spelling alone, is it critical to provide LD services for these children? The issue of prevalence is, in a similar fashion, value-laden. We classify children largely on the basis of continuous variables. Obviously, we are forcing black and white decisions, and in the grey area where many students fall, the decisions become extremely arbitrary. When examining some of the technical issues such as the estimated reliability of the procedure and the use of grade equivalents, one tends to feel that the procedure is inadequate. The real dilemma may be that procedures no more adequate technically than the above procedure are in wide use today. It seerns most appropriate to judge this procedure only in comparison with currently used procedures. One wonders if a technically adequate solution to the problem of LD identification exists. We have tended to believe the major barrier to developing an operational procedure was the definitional problem; however, even with some consensus on definition, a technically acceptable approach may not be available. Given that the proposed procedure uses what might be the best available measurement instruments, a more technically acceptable procedure is unlikely to be identified for general use. Several alternative procedures were suggested by respondents commenting on the proposed regulations; h o w e v e r , these procedures were most often slight variations of the same formula and therefore would present many if not all of the same limitations. The issue of whether or not to use a formulabased procedure in delivering services to LD children will frequently be difficult to resolve. However, formula-based classification could help resolve one of the major concerns of researchers. If routinely used, formula-based procedures could be used as an aspect of the description of the sample studied. An additional advantage of using formula procedures in research is the sophistication we should develop in the application of these procedures. Journal of Learning Disabilities
Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
176 ABOUT THE AUTHORS Louis C. Danielson is currently a statistician in the Division of Innovation and Development at the Bureau of Education for the Handicapped. After receiving his degree in educational psychology from Pennsylvania State University, he worked as a school psychologist and teacher. Jane Bauer, a former educational program specialist in the Division of Innovation and Development, is currently completing her PhD in educational psychology at Stanford University. Requests for reprints should be sent to Dr. Danielson, Bureau of Education for the Handicapped, 400 SW Maryland Ave., Washington, DC. 20202.
REFERENCES Angoff, W.H.: Scales, norms, and equivalent scores. In R.L. Thorndike (Ed.): Educational Measurement (2nd ed.). Washington, D.C.: American Council on Education, 1971. Cronbach, L.J., and Furby, L.: How we should measure
"change" — or should we? Psychological Bulletin, 74,68-80, 1971. Dunn, L.M., and Markwardt, F.C.: Peabody Individual Achievement Test Manual. Circle Pines, Minn.: American Guidance Service, Inc., 1970. Harris, A.: How to Increase Reading Abilities (5th ed.). New York: David McKay, 1970. Sulzbacher, S., and Kenowitz, L.A.: At last, a definition of learning disabilities we can live with? Journal of Learning Disabilities, 10, 8-12, 1977. U.S. Office of Education, Education of Handicapped Children: Proposed Rulemaking. Federal Register 41 (230), 1976. U.S. Office of Education, Education of Handicapped Children: Implementaton of Part B of the Education of the Handicapped Act. Federal Register 42 (163), 1977.a U.S. Office of Education, Assistance to States for Education of Handicapped Children: Procedures for Evaluating Specific Learning Disabilities. Federal Register 42 (250), 1977.b
Volume 11, Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
63
A Formula-based Classification of Learning Disabled Children: An Examination of the Issues
Louis C. Danielson, PhD, and Jane N. Bauer, MA
With the passage of the Education for All Handicapped Children Act, the Bureau of Education of the Handicapped (BEH) was charged with the task of establishing regulations for enactment of the new law. The Federal Register (December 29,1977) reporting the final LD regulations notes that an analysis was made of all comments and testimony regarding the proposed regulations of November 1976. This report, prepared by two BEH scientists, first analyzes the numerous comments, and second describes some BEH in-house research which, along with the comments, prompted the bureau to drop the IQachievement formula in the final regulations. —G.M.S.
W
ith the publication on December 29, 1977, of final learning disabilities (LD) regulations, states and local school districts are left with the task of implementing the procedure for the identification of LD children.* While the application of the proposed LD regulations had been based on a formula, the overwhelming opposition to the use of the formula led to the elimination of both the formula and the specification of a specific severity level for eligibility. During a 120-day review period, the Bureau of Education for the
Handicapped (BEH) undertook some field testing of the formula with data supplied by states and local school districts and also analyzed solicited comments on the adequacy of the formula-based procedure. Some of its findings should be very pertinent as states and local education agencies move to implement the final LD regulations. It is felt that some of the problems and issues related to the use of the federally proposed formula will be typical of other procedures as well. Therefore, this article will outline some generic issues and problems related to the development of operational procedures for identifying LD children.
'The opinions expressed herein do not necessarily reflect the position or policy of the U.S. Office of Education, and no official endorsement by the Office of Education should be inferred.
The final regulations (U.S. Office of Education 1977b) provide that a child has a learning disability under specific conditions:
50
Journal of Learning Disabilities Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
164 (1) The child does not achieve commensurate with his or her age and ability levels in one or more of the areas listed below, when provided with learning experience appropriate for the child's age and ability levels; and (2) The team finds that a child has a severe d i s c r e p a n c y b e t w e e n a c h i e v e m e n t and intellectual ability in one or more of the following areas: (a) oral expression, (b) listening comprehension, (c) written expression, (d) basic reading skill, (e) reading comprehension, (f) mathematics calculations, or, (g) mathematics reasoning. (3) The team may not identify a child as having a specific learning disability if the severe discrepancy between ability and achievement is primarily the result of a visual, hearing, or motor handicap, emotional disturbance, or environmental, cultural, or economic disadvantage.
THE PROPOSED FORMULA The proposed regulation (U.S. Office of Education 1976) had provided the same basic procedure for determining the existence of a specific learning disability; however, they had relied on the use of a formula. Where the final regulations leave the "severe discrepancy" undefined, the proposed regulations defined it as achievement at or below 50$ of the expected achievement in at least one of the several achievement areas, after accounting for age and previous educational experience. In the proposed regulations, the formula for determining the severe discrepancy level (SDL) was: CA / ^ + 0 . 1 7 \ - 2 . 5 = SDL, \300 / where SDL is expressed as a grade equivalent (GE) and CA represents chronological age. As given above, Formula 1 is a transformation of the familiar Harris (1970) formula for estimating a reading age expectancy: Expectancy Age =
2MA
+CA 3
where M A = mental age. This can be converted to a grade expectancy as follows: 2MA+CA _s G E = 3 The SDL which is 50$ of expectancy is then obtained by Formula 2: SDL = ( 2 M y C A ) - 5 » . 5 . This formula is more intuitively appealing than Formula 1 in that it reflects the weighting of MA and CA with conversion to grade equivalents and the subsequent multiplication by .5 to get the 50$ of expected functioning. However, due to objections concerning the use of mental age, MA was replaced by CA -^ in the formula. In addition to this substitution, terms were rearranged to yield Formula 1. The proposed formula was intended to identify children having extreme problems with learning as demonstrated by their achievement, and to provide a common standard for counting children with specific learning disabilities.
ISSUES RELATED Many comments received by BEH during the 120-day period dealt with generic issues concerning the use of a formula. Some comments relating to specific aspects of the proposed formula were interesting because they dealt with potential areas of concern for alternate procedures, i.e., use of IQ scores, use of grade equivalent, etc.
Intelligence Quotient The student's IQ was included in the proposed formula as a measure of ability and was used along with CA to calculate expected achievement. Comments on the inclusion of IQ in the formula raised the issues of validity and reliability and of its appropriateness with children who are learning disabled, of low socioeconomic status (SES), or of preschool age. Since the final regulations seem to require the assessment of intellectual ability, the use of IQs will continue to be a concern of many people.
Volume 11, Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
51
165 Validity of IQ. A number of commentors questioned the use of IQ on the basis of its validity either as a measure of ability or as a measure of potential. Performance on an IQ test may be in part attributable to innate ability; however, since the test depends upon responses which are obviously learned, other characteristics such as test-taking abilities, fatigue, anxiety level, native language, and prior experience with similar tasks affect performance. These characteristics are largely a function of the individual's environment. Therefore, the degree to which an IQ test score can be taken to indicate intelligence may be indeterminate. Another expressed concern is the fact that an IQ score is an aggregate of many discrete abilities and that IQ tests assess different abilities. A child's assessed ability would, therefore, vary depending on the IQ test used. The validity of an IQ test as an estimator of achievement would in turn vary, based on the relationship between those abilities and the achievement in the academic areas of interest. The proposed regulations were also faulted for not attending to the fact that there will be an IQ score error of measurement. The reliability of IQ tests was questioned in general and specifically for preschool children. Appropriateness of IQ. In addition to these general criticisms, there were issues related to the use of IQ scores with specific groups of children: learning disabled, low SES, and high, or low IQ. A substantial proportion of commentors expressed the opinion that being learning disabled would preclude the valid assessment of the child's IQ. Thus, the formula would less likely label this child appropriately as LD. Section 121a.532 of the PL 94-142 regulations addresses this issue (U.S. Office of Education 1977a). The regulations require that a test which purports to measure aptitude accurately reflect the child's aptitude and not his impaired sensory, manual, or speaking skills. Although the regulations seem to prohibit the use of an IQ test which would be affected by a child's learning disability, assessing the child's IQ remains problematic. 52
Many commentors felt the use of IQs in the formula would cause discrimination on the basis of SES. Some felt the procedure would result in the inclusion of too many low SES students in LD programs while others felt it would include too few. A few people felt that children of low SES would do poorly on IQ tests and therefore would be less likely to qualify as LD. One commentor expressed the opinion that since IQ and achievement test performance would be equally affected by SES, the formula could have worked well. The LD regulations state that a child cannot be identified as LD if that severe discrepancy is primarily the result of environmental, cultural, or economic disadvantage. However, our ability to state that a severe discrepancy exists will undoubtedly be much greater than our ability to infer whether it was caused by SES. A few commentors suggested that the formula might identify a greater proportion of children with low IQs than with high IQs. In addition, it is quite possible that there would be fewer high IQ children referred for evaluation, so a formula which is less likely to identify these children combined with fewer children being referred could result in very few above average IQ children being identified as LD. Summary. The major concerns about using IQ in the proposed procedure can be reduced to six points: (1) IQ is not an index of "ability" or "potential." (2) Since intelligence as operationally defined in most tests is an aggregate of many discrete abilities, and tests vary in their composition of these abilities, estimates of ability may vary from test to test. (3) Intelligence as a predictor of achievement will vary from one area to another. (4) The very fact that a child is learning disabled may preclude the valid measurement of intelligence. (5) The fact that a classification procedure relies upon IQ invites cultural discrimination. (6) Incidence figures produced by the formula would not be uniform across IQ levels. Journal of Learning Disabilities
Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
166
Chronological Age Comments related to chronological age primarily concerned the interaction of prevalence with age and the fact that the formula did not account for years in school. Several respondents felt the BEH formula discriminated against older students while others believed the formula discriminated against younger students. It is unlikely that both contentions could be correct since the formula should yield a linear relationship between the SDL and age. Data related to this issue are presented in a subsequent section. Another issue cited, somewhat related to age, contends that the formula would not account for previous educational experience. The formula contained CA, but this obviously translates imperfecdy to years in school by subtracting five since children may vary at least one year in the age they begin school.
Severity Level The preamble to the proposed regulations stated that a severe discrepancy exists when achievement falls at or below 50$ of the child's expected achievement — a figure representing "the level at which a child's educational performance is clearly impaired." The 50$ discrepancy had been incorporated into the formula so that the formula yielded half of expected achievement. This 50$ value would account most directly for the proportion of children who were identified as LD. Many respondents stated that the 50$ figure was arbitrary and were opposed to it. Sulzbacher and Kenowitz (1977) raised two questions concerning the 50$ discrepancy: Is a 50$ discrepancy in math as disabling as a 50$ discrepancy in spelling? Is a 50$ discrepancy for a third grader as deserving of remediation as a 50$ discrepancy in a tenth-grade student? (p.68) LD prevalence would be larger if a value higher than the 50$ level were used. The estimates of prevalence that respondents felt the formula would produce ranged from 1 to 30$. A number of people suggested keeping the 2$ limit
on the number of students a state could count as LD because of the possible inadequacies of the formula. Most of those responding on the basis of prevalence felt the formula would be too restrictive; however, many of these people felt the formula would produce a prevalence figure very near 2$. A number of these individuals suggested that a desirable prevalence figure would be 3 or 4$.
Measurement of Achievement Both the proposed and final regulations specified in part that "a team may determine that a child has a specific learning disability if... the team finds that a child has a severe discrepancy between achievement and intellectual ability." To use the proposed formula, one had to express achievement as a grade equivalent. In implementing the final regulations, some attention should be given to the advantages and disadvantages of various scales. A grade equivalent is a comparatively crude scale and lacks many desirable psychometric properties. For example, a score of 4.2 in reading for a sixthgrade student indicates only very generally the student's standing relative to other students or his mastery of reading. However, grade equivalents are firmly entrenched in the public schools and very familiar to school personnel. Some commentors felt that the focus on standardized norm-referenced testing was undesirable. These respondents supported the use of criterion-referenced instruments and felt that the regulations would discourage the use of these instruments. Further, a number of commentors stated that normed tests are generally unavailable for some of the achievement areas specified in the regulations, and that where tests are available, they are inappropriate for certain groups, e.g., young children and minority children.
Utilization of the Proposed Procedure Both the proposed and the final regulations specify that a child must display a severe
Volume 11, Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
53
167 discrepancy between achievement and ability. The procedure specified by the proposed regulations required that the SDL as determined by the formula be compared to achievement test scores. A child would have been learning disabled according to the formula if achievement ^ SDL, or if achievement - SDL < 0. It is apparent that this decision would have been based on a discrepancy score, which can be quite unreliable (e.g., Cronbach & Furby 1971). The final regulations imply that a discrepancy score would be involved even though it has not been operationalized. One individual responding to the proposed regulations suggested that no more than 25% of those identified as learning disabled by the formula would be so identified in an independent replication of the procedure. In addition, a child would have been required to exhibit the severe discrepancy in only one of the eight achievement areas to qualify as learning disabled. Therefore, the probability of a child exhibiting a severe discrepancy in one of these eight areas due to chance alone could have been substantial. The final regulations now have seven achievement areas. As the proposed regulations were presented, the formula would define a SDL which would be a constant for a child of a given age and IQ and which would then be the same regardless of the deficient achievement area. Several commentors questioned whether the relationship between age, IQ, and achievement is such that the same formula could be used to determine the severe discrepancy in each of the achievement areas. A number of respondents pointed out that there were problems in using the formula at early childhood levels. The proposed regulations pointed out that the formula could result in negative values at these ages although this in itself is not troublesome since they can be translated to grade equivalents, for example, -0.1 equals prekindergarten 0.9. The formula may have had more serious problems at the preschool level; for example, the formula yields a value of -.5 for a four-year-old child with an IQ of 100. If one converts that to an age equivalent by adding 54
5 (the formula originally converted an age expectancy to a grade expectancy by subtracting 5), the severe discrepancy for this child is an age equivalent of 4.5. The four-year-old child would have been LD according to the formula if the child had achievement equal to or less than an age equivalent of 4.5. The formula seems to yield unreasonable values for children younger than five years old since the severe discrepancy is greater than the level at which one would expect the average child to be achieving. This problem becomes more pronounced the lower the age. Some of the respondents referred to the difficulties in measuring achievement at preschool levels. There are obvious problems with reliability and instrumentation at younger ages as well as with the issue of what constitutes a learning disability at the preschool level. For example, the question arises as to what reading achievement is at an early childhood level. A number of commentors expressed the opinion that the regulations could serve to mediate against the early identification and treatment of LD children. A clear concern of a number of respondents involved the problem of distinguishing between measured intelligence as an indication of the child's innate characteristics and achievement as a learned characteristic. This dilemma may be most clear at a preschool level where tests of prereading skills involve the same kinds of tasks as intelligence tests. It had been suggested that the procedure had no established validity; that is, there had been no demonstration of its capability of identifying those children who are in fact LD. Unfortunately, this validation would have been impossible since agreement on which children were or were not LD would have been virtually impossible to obtain. A subsequent section examines the degree to which the proposed formula identified those children who had previously been determined to be LD. The BEH formula could have varied from existing procedures in the restrictiveness of its results; i.e., it may have identified children with Journal of Learning Disabilities
Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
168 TABLE I. Projected incidence figures resulting from the application of the formula to actual data. A
B
Data LD sample Present base size prevalence 1 2 3 4 5 6 7 8 9 10 11 12 13 14
246 101 771 2,428 943 130 180 577 414 82 80 30 63 424
.043 .028 .034
.020
.014 .029 .059 .046 .029
C
D
E
|
F
LD qualifying Proportion of non- Total proportion by form ula Projected LD with IQ ^ 80 qualifying N % prevalence* qualifying by formula (D + E) 93 38 50 50 559 63 1,430 59 439 46 74 57 97 54 363 63 177 43 65 79 59 74 26 87 60 95 279 66
.016 .014 .021
.011
.006 .023 .044 .039 .028
.102 .077 .119 .094
.125 .121 .158 .122 I
*These figures were obtained t •>y multiplying the present prevalence figures by the proportion of the LD students who qualified by the formula. Prevalence refers to the number of LD children divided by the school-age population.
more or less severe handicaps while their characteristics remained the same. Alternatively, the formula may have varied from previous procedures in the definitions it gives, in that its implementation may have constituted a change in definition and, therefore, a change in the characteristics of the children identified. A planned research effort would have been n e c e s s a r y to d e a l w i t h t h e s e i s s u e s comprehensively. Some commentors argued that a change in definition was implicit since the process disorder aspect of identification had not been attended to. Finally, a number of commentors responded that the implementation of the formula would have been difficult since testing would be timeconsuming and teachers were not adequately prepared to use the formula. Another question related to the need for criteria for determining when a child no longer required special services — whether an LD child was ineligible once his achievement level was greater than the SDL defined by the formula.
EMPIRICAL ANALYSES OF FORMULA To examine the applicability of the proposed formula, we obtained information from 14 existing data bases. Since data collection was post hoc and d e p e n d e n t on voluntary cooperation, there are several limitations of the study. Only two data bases (Data Base 4 and 9) included an urban area; a wide range of tests was used to measure achievement; not all children were administered the same number of tests; and in some cases, the population from which the sample was drawn was not known. However, these variations reflect to some degree the varied national procedures. Because of the procedures used to obtain the data bases, these data could not be taken to constitute a sample from any identifiable population. Given the variety and quantity of data, we did feel, however, that if the formula performed poorly on these data, it would not be tenable. On the other hand, if it performed
Volume 11, Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
55
\
169 TABLE II. Percentage of children by IQ level who would be classified asLD by the \ | proposed formula. Data bases for children currently classified as LD IQ 80-90 N
%
91-110 N
%
111 + N
%
1
2
18 44
21 53
75* 68
46 35
24 56
280** 71
22 40
1 25
204f 75
3
Data bases for students with IQ > 80
9
10
11
12
13
10
11
12
13
47 41
32 64
21 75
10 83
29 94
104 40
42 34
34 39
152 39
101 41
25 81
30 71
12 86
30 97
193 24
97 16
54 19
20 20
25 58
8 100
8 80
4 100
1 100
62 9
24 7
16 9
10 5
I
* I Q ^ 84 **IQ = 85-100 f l Q > 101
adequately, we could not presume that it would work as well on other samples. The data are discussed in the following sections according to the question they address.
What Size Prevalence Figures Would Result from the Formula? Many commentors expressed concern over the possible number of children to be identified as LD by the formula because of this information's great fiscal importance, in addition to its effect upon the educational programming and service received by many children. We were interested in ascertaining how the formula might affect the absolute number of children counted and also in anticipating how it might affect whether individual LD children continue to be counted. Prevalence figures (proportion served) for these data bases vary from a low of .014 to a high of .059 (Table I). When the formula was applied to these LD children, between 38$ and 95$ of the children still qualified as LD. For Data Base 4, 56
59$ of the LD children qualified on the basis of the complete agreement of the evaluation team (an alternate method for children to qualify as LD). We calculated a projected prevalence figure by multiplying the proportion of the LD students qualifying under the formula by the present prevalence. This figure represents a minimum prevalence figure because it does not include those children who are not currently identified as LD but would be so identified by the formula. Data in column E reflect the proportion of children who exhibit the "severe discrepancy" but who are not in an LD program. Column F gives the total proportion of children in the data bases that were identified as LD by the formula. Representing an upper limit on the prevalence figure, these figures suggested that if all children in a district with an IQ over 80 were tested in all or nearly all of the eight achievement areas, a prevalence figure as large as 16$ might not be unreasonable. Of course, the prevalence figure Journal of Learning Disabilities
Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
170 I
TABLE
I
as LD by the proposed
III
Percentage of children
.% 9-12 N
%
13 and above N
%
age who would
Data bases for children currently classified as LD
Chronological age
8 and below N
by chronological
be
classified
formula. Data bases for students with IQ > 80
1
2
9
10
11
12
13
68 65
10 29
13 26
17b 85
5b 83
10b 83
17b 100
80 62
14 39
57 30
25° 64
50c 71
14c 88
33c 92
5 45
27a 90
107 61
64 100
d
14 100
d
a
d
2 100
14
10
11
12
13
33e
94 29
15 15
35 34
66 34
57f
190 19
102 14
52 16
138 21
85,77g
75 19
46 19
17 14
64 27
d
10 100
"Grade 1-3 f Grade 4-6 8 Grades 7-9, 10-12
13 and above Below 8 "8-12 d Over 12 b
obtained would probably be dependent on the achievement tests used. Data presented later lead one to suspect this might be the case.
To What Extent Do Estimated Prevalence Figures Vary across IQ Levels? Some commentors felt the BEH formula would not yield uniform prevalence figures across IQ levels. It should be noted that some individuals felt that prevalence figures should not be equal across IQ levels. Data presented in Table II address this issue. Examination of the data on the percentage of LD children qualifying according to the formula across IQ levels suggests that there is no clear variation across IQ levels. As we examined the IQ distribution for the LD children in these data bases it became apparent that the distributions were generally irregular and varied considerably from one data base to another. Since the original distributions were unusual, it is probably unwise to draw any inferences
concerning uniformity of incidence across IQ levels. The form of the distributions may in part be attributable to nonuniform referral and classification procedures and to policies which prohibit children with IQs less than some absolute value from being classified as LD. We felt it might be more appropriate to address the question above by applying the formula to all school-age children in several districts. We first excluded children with IQs less than 80 and then applied the formula to the remaining children. The data show that children of 80 to 90 IQ are much more likely to be determined to be LD by the formula than children with higher IQs (Table II).
To What Extent D o Estimated Prevalence Figures Vary across Age Levels? We found that children were not uniformly distributed across age levels in our LD data bases so again we applied the formula to the entire
Volume 11, Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
57
TABLE IV. Agreement between Wide Range Achievement Test (WRAT) and Peabody Individual Achievement Test (PIAT) subtests in classifying children as LD. Percentage of LP children qualifying by test Test
Reading
Arithmetic
Spelling
PIAT only WRAT only Both PIAT & WRAT Neither N
4.5 7.5 12.0 76.0 67
4.6 22.7 1.5 71.2 66
4.5 15.9 27.3 52.3 44
school-age population with IWs greater than 80 for several districts. Based on the data (TableIII), we feel that children under eight years old are somewhat more likely to be identified as LD by the formula than are older children.
the norms for the math tests differ. There was considerably more overlap between the PIAT and WRAT in the proportion of children found to be LD for spelling and reading (Table V).
The percentage of LD children aged 13 and above who were identified by the formula is somewhat larger than the percentage of those LD children aged 9 through 12. We suspect that this may have resulted because these districts are serving fewer adolescent children who represent the more severely handicapped children.
Does the Proportion of Qualifying Children Vary according to Achievement Area?
To What Extent Does the Effect of the Formula Depend on the Achievement Test Used? Since there were only 66 children who had taken more than one achievement test, the sample for this question was smaller than the sample for the other questions. Data for those 66 children for whom scores on both the Wide Range Achievement Test (WRAT) and the Peabody Individual Achievement Test (PIAT) were available are presented in Table IV. On the basis of the WRAT math score 16 of these children qualified as LD while only four children qualified on the basis of their PIAT math score. Just one child qualified on the basis of both scores. The correlation of the math tests was high, which indicates that these children scored in much the same relative position on each set of tests. These results lead one to suspect that 58
A number of respondents suggested that the formula might qualify relatively larger numbers of children in some achievement areas such as spelling and fewer children in other areas such as math. One is led to believe that this might be the case based on the data presented in Table VI. Among the LD children of Data Base 9, a larger number qualify on the basis of their spelling test on both the WRAT and PIAT. Comparatively fewer children qualify on the basis of their math scores on both the PIAT and the WRAT. On the Stanford Achievement Test, larger numbers qualified on reading recognition and spelling than on both math concepts and math computation. Using standard scores in place of grade equivalents would probably correct this as the outcome is likely related to differences in the variance for these subtests.
DISCUSSION In an attempt to integrate the issues generated with the results of the empirical analyses, the discussion is organized around three questions to Journal of Learning Disabilities
Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
172
TABLE V. Correlations between Peabody Individual Achievement Test and Wide Range Achievement Test subtests from Data Base 9.
WRAT subtests Reading Spelling Math
PIAT subtests Reading Recognition Spelling .91 .63 .86 .79 .61 .50
Mathematics .62 .61 .74
NOTE: Sample sizes ranged from 44 to 67.
determine whether a formula-based procedure should be adopted. (1) Is the formula effective? (2) What would be the impact of the formula? (3) Can the procedure be implemented?
Effectiveness of the Formula For the purpose of this discussion we define effectiveness as the extent to which the procedure meets its goal of identifying those LD children with the most severe problem. Implicit in this goal are two criteria for evaluating the effectiveness of the proposed procedure. (1) Does the procedure reflect what might be taken as the appropriate criterion? (2) Does the procedure identify those children who are the most severely handicapped given that criterion? The procedure identifies as learning disabled those , children at the lower end of the achievement-minus-SDL curve. Because the children who are lowest on this criterion are identified as LD, it obviously identifies the most severely handicapped only if one agrees that the proposed procedure reflects the appropriate criterion. If one accepts the proposition that the learning disabled child is one who exhibits a severe discrepancy b e t w e e n e x p e c t e d achievement and actual achievement, with expected achievement being a function of age and IQ, then the procedure would appear to reflect an appropriate criterion. In general, few
commentors expressed dissatisfaction with the criterion of discrepant achievement. However, some expressed the opinion that the procedure should reflect process disorders. Other commentors questioned the reliance of the procedure upon IQ. Many of these commentors felt that an IQ test is not a valid measure of the ability of LD children. Thus, one might agree conceptually with the notion of abilityachievement discrepancy as a criterion but fault the way it is implemented. If one could identify a group of children who are in fact LD, then the procedure could be directly validated and the appropriateness of the criterion would not be significant; however, the lack of agreement over which children are and are not LD precludes this analysis. In the absence of a generally accepted standard for determining the appropriateness of the proposed procedure, one indication of its validity is the extent to which currently identified LD children are identified by the proposed formula. Across all data bases in our analyses, 58% of the children currently classified as LD were classified LD by the formula. However, the variation between data bases was large, ranging from 38 to 95% of the LD children who would qualify as LD by the formula. Also, the formula should not identify as LD those children who are not LD. Among those children not currently classified as LD, 8 to 12% were classified as LD by the formula.
Volume 11, Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
59
173 TABLE VI. Percentage of students who qualify as LD on specific subtests of the Wide Range Achievement Test, the Peabody Individual Achievement Test, and the Stanford Achievement Test when the formula is applied. Test N LD students — Data Base 9
%
WRAT Reading Spelling Mathematics
82 118 63
23 36 18
PIAT Reading Recognition Spelling Mathematics Reading Comprehension
16 32 9 13
14 30 8
12
Entire student population with IQ ^ 80 — Data Base 11 SAT Reading A 38 9 Reading B 57 14 Reading Comprehension 95 5 Work Study Skills 78 6 Spelling 136 9 Language 104 8 Math concepts 69 4 Math Computation 75 4
technical issues of reliability and scale, which are discussed more fully in a subsequent section. Third, the proposed procedure might differ in substance from procedures currently used. If so, then validity is a major issue. If children labeled LD by one procedure are not so labeled by another procedure, then one or both of the procedures must be inappropriate. Fairness of procedure. One might assume that an appropriate procedure should be fair, according to age and culture. One way to determine fairness is to look at projected prevalence figures across age and IQ. We do not suggest that prevalence should necessarily be equal across IQ, age, and culture; however, in the absence of reasons why it should not be equal, this may be one way of examining fairness. It was found that children with lower IQs seem to be more likely to be identified as LD by the proposed formula than children with higher IQs.
There are several reasons the proposed procedure may not identify all LD children currently served. First, the proposed discrepancy level of 50% may be more severe than many LE As or SEAs are using. Therefore, the procedure identifies only the more severely handicapped of the children currently classified as LD.
The potential impact of using a formula which identified greater proportions of children with lower than with higher IQs might be to cause large numbers of the population identified as LD to be slow learners. There also appears to be a slightly greater probability that younger children would be determined to be LD by the formula than would older children. But, in light of the fact that many commentors were concerned about early identification of the learning disabled, this may not be a troublesome finding. We were unable to address the cultural fairness of the procedure with the data that we obtained. A number of commentors suspected that the procedure would be unfair to some minorities because of its use of IQ.
Second, if the test data used to evaluate the formula were not the same test data used to place the children, then variations between test and retest might lead to different decisions. Since these children are selected on the basis of extreme discrepancies, statistical regression might be anticipated to affect many decisions on a test-re test basis. The problem of test-retest variation in the outcome of applying the proposed procedure is an aspect of the more
Consistency of procedure. An additional measure of effectiveness is the procedure's reliability. If the decisions based on the proposed procedure depended to a large degree upon chance, one could not have seriously considered the procedure viable. The proposed formula involved the use of IQ tests and achievement tests, which can be very reliable. Many school psychologists rely on individual intelligence tests such as the Stanford-Binet or
60
Journal of Learning Disabilities Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
174 the WISC, which are highly reliable. In fact, most of the LD children in our data bases, for whom data were available, were given the StanfordBinet, the WISC, or in some cases, the Slosson. Most frequently the achievement tests used were the WRAT and the PIAT. Reliability for these tests varies by age and by subtest, but as an example, the WISC Full Scale Score reliability is generally above .9 and the median reliability of a PIAT score is .78 (Dunn & Markwardt 1970). The procedure, however, involves the use of a discrepancy score, the reliability of which will be reduced when the tests are positively correlated. As an illustration, given reliability values of .8 and .9 for achievement and IQ, respectively, and a .7 correlation between them, the reliability of the difference would be about .4. If one considers the standard error of measurement in making decisions about individual children, the reliability of the procedure is such that, to be very confident that a child is in fact LD based on his demonstrated discrepancy, the child would have to exhibit an achievement score considerably lower than the SDL. Thus, if one wished to minimize the probability of erroneously classifying children as LD, many children who are LD would not be so identified on the basis of their observed scores. On the other hand, if one wishes to maximize the probability of identifying all children who are in fact LD, one would be likely to identify a substantial number of children who are not LD. Since the procedure identified children who score in the extreme end of the lower tail of the discrepancy curve, the problem of statistical regression must also be considered. This phenomenon, a function of the reliability of the procedure and the distance of the scores from the mean, could result in only a small percentage of those labeled as LD on initial testing, labeled LD at a later testing. One commentor suggested that only about 25% of those children labeled LD on one occasion would again be so labeled in an independent replication of the procedure (Cronbach personal communication, 1977). As was indicated by the data, there may be
some variation in the decision the formula would yield, depending on the particular test used. Although this variation is troublesome, we may be demanding too much from tests if we expect them to be sensitive in the extreme tails. These differences could as well be artifacts of the procedure used to create the grade equivalent scale for the tests. In any case, test variation in terms of the decision they produce would seem to be a useful concern of further research. Another component of the procedure which may affect decision consistency is the use of the grade equivalent. Although the grade equivalent may be a familiar scale to most educators, its use may be a limitation of the proposed formula. As Angoff (1971) very aptly put it: The principal claim that can be made for the grade and age equivalents is that they have a simplicity and directness of meaning in terms of the test user's everyday experience that are not shared by other scales. However, the difficulties and confusions that are attendant on the use of these equivalents would indicate that their simplicity is far more apparent than real and that the truly simple scales may well be those for which there has been no attempt to capitalize on the use of direct meaning. Moreover, while it is possible that direct meaning may be a highly desirable feature in a system of derived scores, the trouble is, as has frequently been pointed out, that users read into such scores more or different meaning than they actually possess, (p. 525) Although not directly addressed in the comments, this problem could be resolved in part by substitution of some other scale, such as standard scores, for the grade equivalent. An extensive review of the methodological limitations of grade equivalents is provided in Angoff and the reader is referred to this source for further elaboration of this issue.
IMPACT OF THE PROPOSED PROCEDURES We define impact as a measure of the consequences of the proposed procedure which are not directly related to the explicit goals of the procedure. We anticipate that the number of LD children that would continue to be labeled LD by
Volume Uy Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
61
175 the proposed procedure would vary substantially from place to place and would be very much dependent on the particular evaluation and placement system in place in an LEA. It seems safe to say that the proposed procedure seems to be more restrictive than many LEAs are currently using. However, variations in the way the procedure might be implemented (e.g., tests used, number of areas tested, proportion of children referred for testing, whether the standard error of measurement is considered) could produce large variation in prevalence. Using this procedure should produce more uniformity in the population of children referred to as learning disabled. Again, the manner of implementing a procedure will affect the consistency of the outcome.
IMPLEMENTATION OF A FORMULA-BASED PROCEDURE The final issue that we address is whether a formula-based procedure can be implemented. Based on data supplied to us, it seems reasonable to conclude that LEAs are already collecting the test information that would be required to use the formula procedure discussed above. This formula is easy to apply; however, there may be some resistance to using this procedure since there were many comments received which were less than favorable concerning the formula.
SUMMARY In sum, the results of the empirical analyses of the formula presented in the proposed regulations suggest that the procedure may have utility. There appear to be some problems with the values the formula produces for preschool children, and the formula may be judged by some to yield prevalence figures that are unacceptably large for slow learners. The whole procedure is plagued by numerous needs for value decisions. For 62
example, if many children are identified on the basis of spelling alone, is it critical to provide LD services for these children? The issue of prevalence is, in a similar fashion, value-laden. We classify children largely on the basis of continuous variables. Obviously, we are forcing black and white decisions, and in the grey area where many students fall, the decisions become extremely arbitrary. When examining some of the technical issues such as the estimated reliability of the procedure and the use of grade equivalents, one tends to feel that the procedure is inadequate. The real dilemma may be that procedures no more adequate technically than the above procedure are in wide use today. It seerns most appropriate to judge this procedure only in comparison with currently used procedures. One wonders if a technically adequate solution to the problem of LD identification exists. We have tended to believe the major barrier to developing an operational procedure was the definitional problem; however, even with some consensus on definition, a technically acceptable approach may not be available. Given that the proposed procedure uses what might be the best available measurement instruments, a more technically acceptable procedure is unlikely to be identified for general use. Several alternative procedures were suggested by respondents commenting on the proposed regulations; h o w e v e r , these procedures were most often slight variations of the same formula and therefore would present many if not all of the same limitations. The issue of whether or not to use a formulabased procedure in delivering services to LD children will frequently be difficult to resolve. However, formula-based classification could help resolve one of the major concerns of researchers. If routinely used, formula-based procedures could be used as an aspect of the description of the sample studied. An additional advantage of using formula procedures in research is the sophistication we should develop in the application of these procedures. Journal of Learning Disabilities
Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
176 ABOUT THE AUTHORS Louis C. Danielson is currently a statistician in the Division of Innovation and Development at the Bureau of Education for the Handicapped. After receiving his degree in educational psychology from Pennsylvania State University, he worked as a school psychologist and teacher. Jane Bauer, a former educational program specialist in the Division of Innovation and Development, is currently completing her PhD in educational psychology at Stanford University. Requests for reprints should be sent to Dr. Danielson, Bureau of Education for the Handicapped, 400 SW Maryland Ave., Washington, DC. 20202.
REFERENCES Angoff, W.H.: Scales, norms, and equivalent scores. In R.L. Thorndike (Ed.): Educational Measurement (2nd ed.). Washington, D.C.: American Council on Education, 1971. Cronbach, L.J., and Furby, L.: How we should measure
"change" — or should we? Psychological Bulletin, 74,68-80, 1971. Dunn, L.M., and Markwardt, F.C.: Peabody Individual Achievement Test Manual. Circle Pines, Minn.: American Guidance Service, Inc., 1970. Harris, A.: How to Increase Reading Abilities (5th ed.). New York: David McKay, 1970. Sulzbacher, S., and Kenowitz, L.A.: At last, a definition of learning disabilities we can live with? Journal of Learning Disabilities, 10, 8-12, 1977. U.S. Office of Education, Education of Handicapped Children: Proposed Rulemaking. Federal Register 41 (230), 1976. U.S. Office of Education, Education of Handicapped Children: Implementaton of Part B of the Education of the Handicapped Act. Federal Register 42 (163), 1977.a U.S. Office of Education, Assistance to States for Education of Handicapped Children: Procedures for Evaluating Specific Learning Disabilities. Federal Register 42 (250), 1977.b
Volume 11, Number 3, March 1978 Downloaded from ldx.sagepub.com at The University of Iowa Libraries on March 18, 2015
63
