MEASURING GENERAL INTELLIGENCE BY INTERVIEW1
By Harvard
Donald Snedden
University
Graduate School
of
Education
By means of a standardized interview, experimentally developed, it was found possible to measure general intelligence with a high degree of reliability ( + .96) and with a substantial degree of validity (+ .82).2 The interview took an average time of 9.33 minutes. The experimental group had a standard deviation of about 30 months mental age, and the criterion used was a composite of the Otis Higher Self Administering Test of Mental Ability, the Terman Group Test of Mental Ability, and Army Alpha Form 9. In spite of these facts, the writer has no desire to recommend technique too heartily?for several reasons. In the first place, the particular interview used in the original investigation cannot very well be used in any but a purely experimental set-up. For the vocational guidance situation, or the employment situation, dif-
this
ferent interviews would have to be standardized. Several workers, including the writer, have made a beginning on this, but all have found that the difficulties involved in constructing plausible yet
discriminating items were all but insuperable. In the first place, items in such a disguised test depend upon their known level of difficulty in comprehension for their validity, and also on the certainty with which the interviewer can tell whether or not the interviewee understands the question. For instance, on a high level, the interviewee might give an adequate answer to "What evidence have you of your stamina?" or "What kind of work have you done involving pertinacity V', but these questions, in the form given, may easily appear silly when presented to an adult in the course of a guidance interview. It has not seemed to be readily possible to construct an employment interview that was not a little more ridiculous than would be desirable. In the second place, the value of this technique depends on another question of rather fundamental importance. Just what do we wish to obtain through the interview? This question may, of course, be answered in various ways, 1
depending
Read at the annual meeting of the National ciation, Atlantic City, Feb. 21, 1930. Snedden, A Study in Disguised Intelligence Bureau of Publication. 1928. 2
on
the
particular
Vocational Guidance AssoTests.
Teachers' College
132
TEE PSYCEOLOGICAL CLINIC
situation in which the interview is used. However, aside from all that, it seems perfectly fair to answer it in general by saying, "We wish to obtain through the interview data, important for our purposes, which cannot be more efficiently and effectively secured in any other way.'' In what specific interview
situations, then, would it be the case intelligence of the subject could not be obthrough regular tests better than through a disguised inter-
that data
on
the verbal
tained view test?
Not, certainly, in the well conducted vocational guidsituation. It is perfectly possible to obtain such test scores as are desired in any vocational guidance organization worthy of the In any situation associated with the school system tests are name. so much a part of that system that no overwhelming difficulty should be found in obtaining such scores as seem pertinent. In the employment situation it is perhaps not so easy to administer tests, but it has been found practicable to give them in most cases. Adult prejudice against tests, while still great enough, is appreciably melting before the enforced modesty of the testers. It is true that in the employment situation, as in all situations where it is desired to obtain a "verbal intelligence" rating, there are several factors to keep in mind. The first of these is the curious lay '' association of the word intelligence'' with '' sanity.'' In somewhat the same way "mental ability" has unfortunate connotations. However, it has been found relatively easy to get parents to do a "vocabulary" test in order to compare the results with tests of their children, while to coax the same parents to co-operate in an "intelligence" test Avould have been very difficult. Johnson O'Conance
avoids the word "test" entirely and substitutes the wholly nonalarming phrase, for instance, "Worksample number 15." The College Entrance Examination Board has undoubtedly avoided a great deal of misinterpretation in naming their general intelligence
nor
test the "Scholastic
of the
Aptitude
Test."
tests have their titles
published accomplished, carefully. may be
Another factor of
as
everyone
some
It is unfortunate that most
clearly displayed. knows, by choosing one's so
Much words
importance, occasionally overlooked, is
index of
"general mental ability" can be reasonably good obtained from the educational record, about which it is always legitimate to ask. The highest grade reached is, of course, of importance, but often of more significance is the age-grade relationship. The that
a
GENERAL INTELLIGENCE
133
fact that individual A left school at the sixth grade means something, but the fact that A was 16 when he finished the sixth grade presents an entirely different pictures from that of A finishing the sixth grade at 12 y^ars of age. Of course, in individual cases the age-grade relationship may be unfair to use as an index because of certain factors, such as much sickness and consequent school absence. is somewhat unusual. possibility, in estimating the intelligence of adult, or child, subjects, is that of adapting the technique reported by Dr. David Levy3 in 1925. This has not been standardized for adults but,
This, however,
Still another
working with over 400 children from five to eleven years old, Dr. Levy obtained by a "directions" technique during the physical examination, intelligence scores which correlated + .94 with StanfordBinet Mental Age. The "doctor" is, generally speaking, more privileged to make strange requests than is the interviewer. A request to "Put your hands way out in front of you, palms downward; then spread out your fingers far apart and turn them around until the palms are upward; then stick out your tongue." (the hardest of Levy's directions) may seem odd, but much of the medical examination is odd. Dr. Levy merely used directions of increasing complexity. A series of five was used, and a system of partial credits for correctness of response gave the score.
If it
were
desired to extend
Dr.
Levy's scale for use with adults it might be improved by including, in addition to "directions" material, a certain amount of vocabulary material, as for example: "Tense your right arm," "In* clinc your head," "Contract your left hand." These might or might not improve the validity of the test. A merit of this type of disguised test is that many employment situations require the physical examination anyway, and by a very small amount of extra time on the part of the examining physician (or an assistant specially trained) the score could be obtained without embarrassment. The general subject of the discussion of which this paper forms a part?"Objective Methods in the Interview"?is of great interest, but Ave should be constantly on our guard lest, in our zeal to make
more
recognized from the s
objective and scientific as
more
ff.
instrument that is generally we do not borrow too much
highly subjective, quantitative techniques.
very
Levy, D. M. A Physical Examination. p. GG9
an
The
case
under discussion
Determining the Mental Ago During of Neurology and PsycJiiatry. 1924. v.
Method for
the
Archives
11.
134
THE PSYCHOLOGICAL CLINIC
(Measurement of Intelligence through Interviews) is one in point. The writer found it possible, in his own experimental work, to do this job of interview intelligence measurement reasonably well. The
subjects test was
idea they were being tested and the validity of the This is an interesting fact, but from this not at all forced to the conclusion that this is, in general,
had
no
good enough.
fact we are the most effective way to measure intelligence. In a very real sense the interview has a residual function. Left to the interview are all those items of relative intangibility which are not measurable in a more direct and a less subjective manner. Generally speaking, before the days of intelligence tests one of the interviewer's chief How concerns was to estimate the intelligence of the interviewee. well this can be done, or rather how nearly impossible it is to do, is, or should be, a matter of common knowledge among workers in this field.4 But, there have been developed instruments for measuring intelligence in a rather reliable manner, and if a knowledge of the amount of general (Verbal) intelligence is called for, it should means be obtained by an undisguised test, or by the same disguised merely in name, as in Johnson O'Connor's worksamples. Only where it is altogether undesirable or impossible to use such well standardized objective measures should consideration be given to working out and using either (a) an extension of Levy's technique, which has the merit of being rather generally applicable since the routine physical examination is a part of many guidance and placement situations, or (b) a standardized interview based on the writer's technique, or (c) illuminating (when checked) indices of "verbal intelligence" based on the age-grade relationship.
by
all
test
4
(The typical correlation, between interview estimates of intelligence fairly reliable test results, varies, depending on various circumstances, around -f- .20). and
By Harvard
Donald Snedden
University
Graduate School
of
Education
By means of a standardized interview, experimentally developed, it was found possible to measure general intelligence with a high degree of reliability ( + .96) and with a substantial degree of validity (+ .82).2 The interview took an average time of 9.33 minutes. The experimental group had a standard deviation of about 30 months mental age, and the criterion used was a composite of the Otis Higher Self Administering Test of Mental Ability, the Terman Group Test of Mental Ability, and Army Alpha Form 9. In spite of these facts, the writer has no desire to recommend technique too heartily?for several reasons. In the first place, the particular interview used in the original investigation cannot very well be used in any but a purely experimental set-up. For the vocational guidance situation, or the employment situation, dif-
this
ferent interviews would have to be standardized. Several workers, including the writer, have made a beginning on this, but all have found that the difficulties involved in constructing plausible yet
discriminating items were all but insuperable. In the first place, items in such a disguised test depend upon their known level of difficulty in comprehension for their validity, and also on the certainty with which the interviewer can tell whether or not the interviewee understands the question. For instance, on a high level, the interviewee might give an adequate answer to "What evidence have you of your stamina?" or "What kind of work have you done involving pertinacity V', but these questions, in the form given, may easily appear silly when presented to an adult in the course of a guidance interview. It has not seemed to be readily possible to construct an employment interview that was not a little more ridiculous than would be desirable. In the second place, the value of this technique depends on another question of rather fundamental importance. Just what do we wish to obtain through the interview? This question may, of course, be answered in various ways, 1
depending
Read at the annual meeting of the National ciation, Atlantic City, Feb. 21, 1930. Snedden, A Study in Disguised Intelligence Bureau of Publication. 1928. 2
on
the
particular
Vocational Guidance AssoTests.
Teachers' College
132
TEE PSYCEOLOGICAL CLINIC
situation in which the interview is used. However, aside from all that, it seems perfectly fair to answer it in general by saying, "We wish to obtain through the interview data, important for our purposes, which cannot be more efficiently and effectively secured in any other way.'' In what specific interview
situations, then, would it be the case intelligence of the subject could not be obthrough regular tests better than through a disguised inter-
that data
on
the verbal
tained view test?
Not, certainly, in the well conducted vocational guidsituation. It is perfectly possible to obtain such test scores as are desired in any vocational guidance organization worthy of the In any situation associated with the school system tests are name. so much a part of that system that no overwhelming difficulty should be found in obtaining such scores as seem pertinent. In the employment situation it is perhaps not so easy to administer tests, but it has been found practicable to give them in most cases. Adult prejudice against tests, while still great enough, is appreciably melting before the enforced modesty of the testers. It is true that in the employment situation, as in all situations where it is desired to obtain a "verbal intelligence" rating, there are several factors to keep in mind. The first of these is the curious lay '' association of the word intelligence'' with '' sanity.'' In somewhat the same way "mental ability" has unfortunate connotations. However, it has been found relatively easy to get parents to do a "vocabulary" test in order to compare the results with tests of their children, while to coax the same parents to co-operate in an "intelligence" test Avould have been very difficult. Johnson O'Conance
avoids the word "test" entirely and substitutes the wholly nonalarming phrase, for instance, "Worksample number 15." The College Entrance Examination Board has undoubtedly avoided a great deal of misinterpretation in naming their general intelligence
nor
test the "Scholastic
of the
Aptitude
Test."
tests have their titles
published accomplished, carefully. may be
Another factor of
as
everyone
some
It is unfortunate that most
clearly displayed. knows, by choosing one's so
Much words
importance, occasionally overlooked, is
index of
"general mental ability" can be reasonably good obtained from the educational record, about which it is always legitimate to ask. The highest grade reached is, of course, of importance, but often of more significance is the age-grade relationship. The that
a
GENERAL INTELLIGENCE
133
fact that individual A left school at the sixth grade means something, but the fact that A was 16 when he finished the sixth grade presents an entirely different pictures from that of A finishing the sixth grade at 12 y^ars of age. Of course, in individual cases the age-grade relationship may be unfair to use as an index because of certain factors, such as much sickness and consequent school absence. is somewhat unusual. possibility, in estimating the intelligence of adult, or child, subjects, is that of adapting the technique reported by Dr. David Levy3 in 1925. This has not been standardized for adults but,
This, however,
Still another
working with over 400 children from five to eleven years old, Dr. Levy obtained by a "directions" technique during the physical examination, intelligence scores which correlated + .94 with StanfordBinet Mental Age. The "doctor" is, generally speaking, more privileged to make strange requests than is the interviewer. A request to "Put your hands way out in front of you, palms downward; then spread out your fingers far apart and turn them around until the palms are upward; then stick out your tongue." (the hardest of Levy's directions) may seem odd, but much of the medical examination is odd. Dr. Levy merely used directions of increasing complexity. A series of five was used, and a system of partial credits for correctness of response gave the score.
If it
were
desired to extend
Dr.
Levy's scale for use with adults it might be improved by including, in addition to "directions" material, a certain amount of vocabulary material, as for example: "Tense your right arm," "In* clinc your head," "Contract your left hand." These might or might not improve the validity of the test. A merit of this type of disguised test is that many employment situations require the physical examination anyway, and by a very small amount of extra time on the part of the examining physician (or an assistant specially trained) the score could be obtained without embarrassment. The general subject of the discussion of which this paper forms a part?"Objective Methods in the Interview"?is of great interest, but Ave should be constantly on our guard lest, in our zeal to make
more
recognized from the s
objective and scientific as
more
ff.
instrument that is generally we do not borrow too much
highly subjective, quantitative techniques.
very
Levy, D. M. A Physical Examination. p. GG9
an
The
case
under discussion
Determining the Mental Ago During of Neurology and PsycJiiatry. 1924. v.
Method for
the
Archives
11.
134
THE PSYCHOLOGICAL CLINIC
(Measurement of Intelligence through Interviews) is one in point. The writer found it possible, in his own experimental work, to do this job of interview intelligence measurement reasonably well. The
subjects test was
idea they were being tested and the validity of the This is an interesting fact, but from this not at all forced to the conclusion that this is, in general,
had
no
good enough.
fact we are the most effective way to measure intelligence. In a very real sense the interview has a residual function. Left to the interview are all those items of relative intangibility which are not measurable in a more direct and a less subjective manner. Generally speaking, before the days of intelligence tests one of the interviewer's chief How concerns was to estimate the intelligence of the interviewee. well this can be done, or rather how nearly impossible it is to do, is, or should be, a matter of common knowledge among workers in this field.4 But, there have been developed instruments for measuring intelligence in a rather reliable manner, and if a knowledge of the amount of general (Verbal) intelligence is called for, it should means be obtained by an undisguised test, or by the same disguised merely in name, as in Johnson O'Connor's worksamples. Only where it is altogether undesirable or impossible to use such well standardized objective measures should consideration be given to working out and using either (a) an extension of Levy's technique, which has the merit of being rather generally applicable since the routine physical examination is a part of many guidance and placement situations, or (b) a standardized interview based on the writer's technique, or (c) illuminating (when checked) indices of "verbal intelligence" based on the age-grade relationship.
by
all
test
4
(The typical correlation, between interview estimates of intelligence fairly reliable test results, varies, depending on various circumstances, around -f- .20). and
