ACCURACY OF PUPIL REPORTING. By J. C.
K. J.
Hoke,
Teachers
College,
Lewis,
J. B.
Welles,
New York
City,
and
G. M.
Wilson,
N. Y.
the past few years many studies in education, particularly those concerning retardation and promotion, have been based upon the answers given by children to questions asked by the investigators. While this method of collecting data has been known to be
During
no one has attempted to see how much the error was nor in which direction it lay, that is, whether it was in favor of the school examined or not. In order to get some idea of this problem, a study
inaccurate,
was made among the school children of a city near New York,* who had attended the schools for the past six years and whose cumulative record cards for this period were complete. The children
selected group in that they had spent their entire school life, six years in extent, in this particular school system. It was found however on careful examination that the selection with regard to race and financial circumstances was really a random one. The
were a
grade distribution was normal. The grade three 9, grade four 33, grade five 54, grade six 74, grade seven 5, grade eight 1, total 176. The distribution by ages was,? 5 nine years old, 41 ten years old, 71 eleven years old, 42 twelve years old, 12 thirteen years old, 4 fourteen, 1 fifteen; total 176. There were 63 boys and 93 girls, total 176. The pupils were asked to fill out the following blank, and the age distribution
as
well
distribution by grades
returns
were
the
as
was,?
secured from 137 of the 176 children in the six year
group:
Name 1. Have you 2. What
ever
grade
3. Have you 4. In what
or
skipped grades
stayed
grade
or
two
a
grade?
have you or
grades
skipped?
more years
have you
in any
stayed
grade?
more
than
a
year?
* The authors desire to recognize their indebtedness to Dr. G. D. Strayer, Department of School Administration, Teachers College, under whose direction this study was undertaken, and whose co-operation made it possible. The original data for this study are on file in Dr. Strayer'a office.
(135)
THE PSYCHOLOGICAL CLINIC.
136
To avoid too much disturbance and to have the questions answered under normal conditions, the teachers gave them out and returned them to the investigators. In one school the children were assisted
by the teachers in recalling their school-life, and in another school the children had been cooperating with the principal in the correction of their record cards just previous to this study. These facts would tend to make a slightly larger percentage of the answers correct. The investigators carefully examined the cumulative record card of each pupil and checked the answers accordingly. In checking up the answers, if the pupil said that he was retarded one year in the second grade, but was recorded as having been retarded one year in the first grade, his answer was accepted as correct. This liberal
construction tends further to make the percentage of correct answers large. If the pupil reported an answer in any other way than
too
that shown
it
above,
was
considered incorrect.
Table I shows the
results in detail. TABLE I. ANSWERS OF
Normal Children
o
Say
Say
ret.
accel.
SCHOOL CHILDREN.
Accelerated
Incorrect
O
137
Retarded
Incorrect h
o
O
Say normal
Incorrect a
Say
Say
O
nor-
accel.
o
mal 51
31
27
1 yr. 2 yrs. too too
few
No ans.
few
17
A double
scoring was used in the case of double retardations promotions. By this method the actual number of pupils' answers will be found to exceed the number of pupils by eleven. Reading the table from the left we find that of the normal children fifty-one gave correct answers, four incorrect; two of these saying that they had been retarded and two accelerated. Of the accelerated children three answered correctly and one incorrectly, the one saying that he was normal. Of the retarded children, we find that thirtyone answers were correct, and of those answering incorrectly, twentyseven said that they were normal, seventeen that they were accelerated. Four reported that they had been retarded, but reported and
one
year less than
the records showed and four two years less
Finally we find that six gave no answer at all. Reducing this table to a percentage basis, and charting it, we find that 93 per cent of the children who were normal answered correctly, than the records.
ACCURACY OF PUPIL REPORTING.
137
while 3.5 per cent said that they were retarded. Among the accelerated children the small number of cases distorted the percentages; one child said that he was normal. The most significant facts were found in the answers of the retarded children. Sixty-five per cent of the answers were incorrect. This percentage was distributed among the different mistakes as follows: 30 per cent reported normal progress, 19 per cent said they were accelerated, 4.5 per cent reported one year less than they bad actually been retarded. Another 4.5 per cent reported twTo years less retardation than
they had actually had, and 7 per cent gave no It may be evident therefore that normal and accelerated pupils usually know what happened to them, but retarded pupils do not recall readily their past life in school. The fact is brought
answer.
home to every school man that no accurate study of grade progress can be made without fully kept cumulative record cards.
straight
Let
us
apply
the results of this
inquiry
to
a
well known
study
based upon the answers of pupils as to retardation or failure,?" The Incidence of Retardation" by Dr. Louis B. Blan (Teachers College, This
1911).
appealed
splendid study;
but their
described above for
checking
to the writers as a
scientific interest led them
to the
plan
as
up, for proving or disproving the results. Blan studied 4579 children as follows: a New York City district, 1312 cases; Elizabeth, N. J., 1088 cases; Paterson, N. J., 1246 cases; East Orange, N. J., 448 cases; Plainfield, N. J., 485 cases. By a method identical with our own as shown by an examination of Blan's paper, pp. 22, 32, and 36, the 4579 selected pupils in the
five cities
distribution of the 3947 cases These were reduced to a percentage basis for each This group and then the median of the five groups calculated. median for the fifth grade pupils is: were
studied
as to
the
grade
of retardation.
Per cent of failures in the fifth "
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
grade
11.2
fourth
"
9.2
third
"
7.2
second
"
6.6
first
"
7.2
study tends to show that in the lower grades these figsmall, it is only fair to call attention to the evidence in Blan's own study tending to support the conclusion which we reach. We are surprised that so careful a student should have neglected Since
ures are
our
much too
THE PSYCHOLOGICAL CLINIC.
138
evidence, tending as it does to prove the unreliability of a pupil's as to his own failure, a conclusion which if substantiated would invalidate his entire study. Blan says (p. 31): "Each pupil was asked to state in what grade or grades he had been kept back for a second term. In the event of the pupil's inability to remember accurately, a note was made of such inability and recourse was had to the record card filed in the office of the principal. Such action was necessary in 32 out of 1312 cases and the writer was particularly fortunate in obtaining this
report
full records of same." The above quotation shows how little use was made of the cumulative record cards in checking up pupils' answers. Blan (p. 52) notes that the higher the grade the fewer failures there are reported for lower grades, but it does not seem to occur to him that this failure to report may have been due to the inability of pupils to remember the failures that occurred so many years before. Blan says: "In fine, then, it may be said that the fourth or fifth grade pupil is left back in the third grade more than twice as often as the eighth or seventh grade pupil." In Plainfield, N. J., there were cumulative record cards which Blan could have used, but of which he made little use. It appears however that they had some effect in helping the memory of pupils. The 485 pupils in this system reported 616 failures, or a larger proportion than in any of the other systems involved in the study. The first grade failures reported in this system are from five to eight times the median per cent of the other systems, and range from 30 to 39.2 per cent. In connection with this system Blan says: "The
writer, however, visited each class and eighth grades and questioned other cities.
.
.
.
the
Their memory
of the
fifth, sixth, seventh, pupils individually as in the seemed to wane only in the
room
Wherever it was found that primary grades. misstated the facts, their records as read from pupils deliberately cards were individual history invariably somewhat worse than they For to admit. cared example, when pupils replied that they were number of times, on checking up their statements left back a given in that no case were they left back less than they stated. it was found in On the other hand, quite a number of instances their history cards showed one or more retardations above the number admitted by case
of the
.
.
.
them in class." The values to be attached to the expressions "somewhat" and a number" in the above quotation can be best estimated
"quite
from the details of table I of this 65 per cent of
pupils'
study.
answers as to
It must be evident that if
retardation
are
incorrect,
the
ACCURACY OF PUPIL REPORTING.
only profitable
way in which to
study
this
139
is
question
through
an
examination of the individual cumulative record cards. Let us proceed a step further by distributing the retardations as actually occurring and as reported by the children, and then apply these facts to Blan's study. Table II shows the distribution of and reported unreported failures, and the total of the whole number
actually occurring. TABLE II.
Number of Retardations not reported
Number of Retardations reported
Grade.
V
IV III II
IV.
I
IV III II
Total
I
V
14
1
6
7
1
3
5
12
1
1
16
22
2
4
13
III.
Retardations.
IV III II
I
4
6
14 21
2
2
21
3
34
5 13
Taking the first line, table II reads, beginning at the left, the fifth grade pupils reported 1 retardation in grade five, 3 retardations in grade four, 3 in grade three, 6 in grade two, and 7 in grade one; and they failed to report 1 retardation in grade four, 3 in grade grade two, and 14 in grade one; the distribution of total failures among fifth grade pupils being 1 in grade five, 4 in grade four, 6 in grade three, 14 in grade two, and 21 in grade one. The next line of table II shows the distribution of fourth grade retardations; the last line, the distribution of third grade retardations. It will be seen from the table that fourth grade pupils report more inaccurately than fifth grade pupils, and that the third grade pupils do poorest of all. The only place where this table overlaps Blan's is in the fifth It will be interesting therefore to calculate the incidence of year. retardation for the fifth grade and compare the results with the figures in Blan's table. Since Blan used pupils' reports, however, it wall be proper for us to use pupils' reports in making the comparison. This comparison is shown in table III. This table is derived directly from table II. The 4.7 per cent under grade I of table III, simply means that 7 in table II is 4.7 per cent of 148,?148 being the total number of cases involved, including normal, accelerated, and retarded, each retardation being
three,
8 in
counted as of 148, etc.
a
separate
case.
Likewise 6 in table II is 4.0 per cent
140
THE PSYCHOLOGICAL CLINIC. TABLE III.
SHOWING THE INCIDENCE OF RETARDATION FOR FIFTH GRADE PUPILS ACCORDING TO THEIR OWN REPORTS.
Grade
As per
IV
in
II
.7
2.0
2.0
4.0
4.7
11.2
9.2
7.2
6.6
7.2
pupils' reports.
Blan, based
on
pupils' reports.
Table IV is derived from table II in the same maimer, but the That is, 21 is 14.2 per cent of 148; totals for grade V are used. 14 is 9.5 per cent of 148, etc. The fifth grade incidence of retardation thus derived is based upon the total retardations, the retardations reported by the children plus those not reported. The figures showing the corrected incidence of retardation for Blan's study are TABLE IV.
SHOWING THE CORRECT FIFTH GRADE INCIDENCE OF RETARDATION FOR THIS
STUDY, AND THE CORRECTED INCIDENCE FOR BLAN's STUDY.
IV
III
II
.7
2.7
4.1
9.5
14.2
11.2
12.4
14.7
15.7
21.7
Grade
This
study
as
per record cards
Blan,allowing for
errors
of
pupils' reports
by simple proportion in each case, and assume that the reporting of pupils in his study was inaccurate to the same degree and in the same direction as the reporting of the children in this study. If this assumption is correct, and Blan's own study bears evidence that it is, the conclusions reached by Blan are equally unreliable. The situation as to failure in upper as compared with lower grades is exactly reversed.
secured
It is not necessary to pursue the discussion further. this study is not conclusive, it points strongly towards the
While
following
conclusions:
First.?Pupils' reports likely to be very inaccurate.
as
Second.?The inaccuracies and where failure is involved.
to progress are
through
greatest
the
grades
in the lower
are
grades
NEWS AND COMMENT.
141
to the progress of children through the reports of children are therefore more or less inaccurate and may be directly contrary to the facts. Fourth.?As the cumulative record card is being moie and more generally introduced, we are justified in insisting that future studies as to the progress of children through the grades shall be based upon the evidence furnished by such card. Third.?Conclusions
grades when based
as
upon the