Public Health
(1991), 105, 139-144
© The Society of Public Health, 199l
Comparison of the A u t o m a t e d Vision Screening Test to the Snellen Test R. Gofin and M. Falk
Department of Social Medicine--Hadassah Medical Organization and the Hebrew University--Hadassah School of Public Health and Community Medicine, P.O,B. 1172, Jerusalem 91010
The comparison of an automatic vision screening machine using the Landolt rings and the usual Snellen Chart was carried out among t23 second grade and 149 fifth grade students in an elementary school in Jerusalem. The sensitivity of the test for a cut-off point of />6/12 according to the Snellen test was 41.7% (CI=16.5-71.4) and the specificity was 86.5% (CI 78.1-92.2) for second graders. For fifth graders the values were 50.0% (CI = 20.1-79.9) and 90.6% (CI 83.7-94.8) respectively. Diagnosis by a specialist decreased the number of 'false negatives' and confirmed the pathological cases. The automatic test was well accepted by the children. Though more time is needed for assessment using the automatic test, its advantage is that no professional staff are required. Vision screening is an established practice in school health care. It aims at the early detection of refractive errors, amblyopia and strabismus. An accepted method for screening for visual acuity is the Snellen chart. ~This test is administered to young children using the illiterate E chart and for children older than seven years the letter or n u m b e r chart is used. The test is considered to be reliable and valid for testing the retinal resolving power. It is used as the 'gold standard' before other tests are introduced into practice. 2 With the advancement of technology, new automated screening methods have been developed. One o f the aims of automated assessment is to release the school nurses from the screening procedure and introduce a test with higher accuracy. The present paper aims at studying the validity of an automated procedure of screening visual acuity a m o n g school children. Before introducing the study, the principles for use o f screening as defined by Cochrane and Holland 3 are outlined. The criteria for the test should be: (1) Simplicity. It should be quick and easy to interpret and capable o f being performed by paramedical and other personnel. (2) Acceptability. It should be acceptable to the subjects. (3) Accuracy. It must give a true measurement of the attribute under study. (4) Cost. The cost of screening should be considered in relation to the benefits resulting from early detection of the disease. (5) Precision. The test should give consistent results in repeated trials, bearing in mind that the consistency o f results is dependent upon subject variability, observer variability and variability of the equipment used. (6) Sensitivity. The test must be capable o f giving a positive finding when the individual being screened has the disease or abnormality under investigation. (7) Specificity. It must be possible to obtain a negative result when the subject does not have the condition. Correspondence: Rosa Gofin, M.D., M.P.H., Department of Social Medicine, Hadassah School of Public Health and Community Medicine, P.O.B. 1172, Jerusalem 91010, Israel.
R. Gofin and M. Falk
140
The concept of 'yield', defined as the number of new, previously unsuspected, cases detected per 100 cases screened, was added by Rose n to the above.
Population and Methods The study population consisted of 123 children attending second grade and 149 children in fifth grade in two elementary schools in a middle class neighbourhood of West Jerusalem. The mean ages were 7.8 and 10.8 years, respectively. Children varied with respect to their ethnical background, as determined by region of birth o f the mother: 67% were Israeli born, 20.1% from Asian-African countries and 12.9% from European American countries. There were 14.1% of mothers with up to nine years of education, 56.5% had 10 12 years of education and 29.4% had 13 or more years of education. The vision screening was administered by three trained school nurses using the Snellen chart and the Canon CV-10 machine. Before the administration of the test, the nurse gave an explanation to each class about the two methods to be used. Each child had both tests on the same day. The order in which the classes performed the test was randomly selected and nurses performing the test were randomly allocated to each class. Results of the first test administered were not known to the nurse administering the second test. F o r the Snellen test the child stood five metres (six yards) from the chart. This distance was chosen to be comparable to the one used by the Canon test. Each eye was tested separately. The C a n o n machine uses a broken circle, the Landolt rings, 5 as the test method. The child uses a joystick to indicate in which segment the gap of the circle is located. Both eyes were checked. In this test for monocular vision, there is no need to cover one eye. A synthesized voice guides the examinee through the procedure. Results of the test are automatically printed when the test is over. Children who had a vision of >~6/12 in either eye or two lines difference according to the Snellen test were referred to the specialist. The duration of the screening procedure was measured by a stop-watch for the Snellen test and automatically by the built-in clock in the C a n o n machine. Students from the sixth grade were responsible for measuring the duration of the Snellen test and they also helped the nurse to organize the screening activities. At the end of both procedures, children were asked about the difficulties and their satisfaction with each test. Data were analysed using SPSSX. 6 To test the validity o f the new screening method, the Snellen test was used as a 'gold standard'. The results were dichotomized as 'normal' (a value of 6/6, 6/9) and ' a b n o r m a l ' (a value o f >~6/12). Different cut-off points were selected for the C a n o n machine. The calculation o f sensitivity and specificity was as follows:
Snellen abnormal normal ~>6/12 ~ 6/12 for either eye according to the Snellen test. Five (4.1%) o f the students in the second grade and seven (4.7%) in the fifth grade were already known. The comparison between both screening tests is seen in Tables I and II. The specificity of the test (those identified as normal by the C a n o n test, a m o n g normal children by the Snellen test), increased f r o m 64.4% for a cut-off point o f >~6/10 to 86.5% for a cut-off point o f ~>6/12 or >~6/15 a m o n g children in the second grades, and from 77.2% to 90.6% and 91.3% for the same cut-off points a m o n g fifth graders. Concurrently, between 14-36% of second graders and 9 23% o f fifth graders will be over-referred (false positives). Sensitivity, or those who will be identified as pathological a m o n g children with vision problems by the Table I
Comparison of the automated vision screening test with the Snellen chart as a 'gold standard' Second grade
Fifth grade
Snellen
Snellen
>~6/12 ~~6/12 Canon />6/10
5 7 12
14 90 104
~>6/12 46/9 5 5 l0
12 115 127
Sensitivity
5 12-41.7 (16.5-71.4)
5 10-50.0
(20.1-79.9)
Specificity
oo 1 ~ = 8 6 . 5 (78.1-92.2)
ll5 127-""~u°~
(83.7 94.8)
(c.i. in brackets)
Table II
/> 6/10 >~6/12 ~>6/15
Sensitivity and specificity of the automated vision screening test compared to the Snellen Chart test using three different cut-off points Second grade Sensitivity (c.i.) Specificity (c.i.)
Fifth grade Sensitivity (c.i.) Specificity (c.i.)
66.7 (35.4-88.7) 41.7(16.5 71.4) 25.0(6.7-57.2)
80.0 (44.246.5) 50.0(20.1-79.9) 30.0(8.1-64.6)
64.4 (54.4-73.4) 86.5(78.1-92.2) 86.5(78.1-92.2)
77.2 (68.7-83.9) 90.6(83.7 94.8) 91,3(84.7 95.4)
The scale for the Snellen test is 6/6, 6/9, 6/12, 6/15, 6/30, 6/60. The scale for the Canon test is 6/6, 6/7.5, 6/10, 6/ 12, 6/15, 6/20, 6/30, 6/60.
142
R. Gofin and M. Falk Table III
Reported characteristics of the screening tests by grade Grade
Which test was easier? Snellen Canon Both Don't know Total Which test did you like more? Snellen Canon Both Don't know Total
Second n %
n
48 49 17 2
70 44 26 2
41.4 42.2 14.7 1.7
Fifth % 49.3 31.0 18.3 1.4
116 100.0 142 100.0 17 86 12 -
14.8 12 7 4 . 8 117 10.4 11 3
8.4 81.8 7.7 2.1
115 100.0 143 100.0
Snellen test, was low: 66.7% for a cut-off point of ~>6/10 which decreased to 41.7% and 25.0% for cut-off points of >~6/12 and >~6/15 a m o n g children in the second grade. The sensitivity was higher among fifth graders: 80.0%, 50% and 30%, respectively. Twenty-three children were referred for further assessment. A m o n g the 13 children in the second grade, three obtained glasses (two of them also tested >~6/12 by the Canon test), four were normal (they also screened normal by the Canon test) and no answer was obtained for six of the children (two of them left school and could not be located). A m o n g the latter children, one child did not know how to operate the Canon machine, four were normal and one had a value o f / > 6 / 1 2 . A m o n g the ten children in the fifth grade who were referred, five obtained glasses (four were also screened positive by the C a n o n test); three were normal (two were normal by the Canon test) and for two cases no answer was obtained (one of them left the area); both screened positive by the Canon test. When re-tested 27 (87%) children from the second grade and 25 (83%) from the fifth grade screened normal again. The remaining nine children were screened abnormal the first time and normal the second. Three in each grade screened 6/60 the first time for the first eye examined. The easiness and satisfaction of both procedures are seen in Table Ill. The proportion of second grade children who considered the test easy was similar for both the Snellen and the Canon tests. Older children in the fifth grade reported the Snellen procedure to be easier in higher proportion. However, children in the second grade reported an easier test for the last test performed, whether Snellen or Canon. Both groups of children reported that they preferred the Canon to the Snellen test: 74.8% a m o n g second graders and 81.8% a m o n g fifth graders. The Snellen test took significantly less time than the Canon procedure in both the second and fifth grades (Table IV). Children in the fifth grade did the test more quickly than younger children. The time taken to screen 1,000 students in second grade can be estimated
Comparison of the Automated Vision Screening Test to the Snellen Test Table IV
143
Time (in seconds) taken to administer the screening tests by grade*
Test Snellen Canon
Second grade (n - 118) Mean + s.d.
Fifth grade (n = 144) Mean ± s.d.
47.0 ± 22.4 82.9 zk34.9
39.7 + 17 77.9 ± 28.3
* Student t test P
(1991), 105, 139-144
© The Society of Public Health, 199l
Comparison of the A u t o m a t e d Vision Screening Test to the Snellen Test R. Gofin and M. Falk
Department of Social Medicine--Hadassah Medical Organization and the Hebrew University--Hadassah School of Public Health and Community Medicine, P.O,B. 1172, Jerusalem 91010
The comparison of an automatic vision screening machine using the Landolt rings and the usual Snellen Chart was carried out among t23 second grade and 149 fifth grade students in an elementary school in Jerusalem. The sensitivity of the test for a cut-off point of />6/12 according to the Snellen test was 41.7% (CI=16.5-71.4) and the specificity was 86.5% (CI 78.1-92.2) for second graders. For fifth graders the values were 50.0% (CI = 20.1-79.9) and 90.6% (CI 83.7-94.8) respectively. Diagnosis by a specialist decreased the number of 'false negatives' and confirmed the pathological cases. The automatic test was well accepted by the children. Though more time is needed for assessment using the automatic test, its advantage is that no professional staff are required. Vision screening is an established practice in school health care. It aims at the early detection of refractive errors, amblyopia and strabismus. An accepted method for screening for visual acuity is the Snellen chart. ~This test is administered to young children using the illiterate E chart and for children older than seven years the letter or n u m b e r chart is used. The test is considered to be reliable and valid for testing the retinal resolving power. It is used as the 'gold standard' before other tests are introduced into practice. 2 With the advancement of technology, new automated screening methods have been developed. One o f the aims of automated assessment is to release the school nurses from the screening procedure and introduce a test with higher accuracy. The present paper aims at studying the validity of an automated procedure of screening visual acuity a m o n g school children. Before introducing the study, the principles for use o f screening as defined by Cochrane and Holland 3 are outlined. The criteria for the test should be: (1) Simplicity. It should be quick and easy to interpret and capable o f being performed by paramedical and other personnel. (2) Acceptability. It should be acceptable to the subjects. (3) Accuracy. It must give a true measurement of the attribute under study. (4) Cost. The cost of screening should be considered in relation to the benefits resulting from early detection of the disease. (5) Precision. The test should give consistent results in repeated trials, bearing in mind that the consistency o f results is dependent upon subject variability, observer variability and variability of the equipment used. (6) Sensitivity. The test must be capable o f giving a positive finding when the individual being screened has the disease or abnormality under investigation. (7) Specificity. It must be possible to obtain a negative result when the subject does not have the condition. Correspondence: Rosa Gofin, M.D., M.P.H., Department of Social Medicine, Hadassah School of Public Health and Community Medicine, P.O.B. 1172, Jerusalem 91010, Israel.
R. Gofin and M. Falk
140
The concept of 'yield', defined as the number of new, previously unsuspected, cases detected per 100 cases screened, was added by Rose n to the above.
Population and Methods The study population consisted of 123 children attending second grade and 149 children in fifth grade in two elementary schools in a middle class neighbourhood of West Jerusalem. The mean ages were 7.8 and 10.8 years, respectively. Children varied with respect to their ethnical background, as determined by region of birth o f the mother: 67% were Israeli born, 20.1% from Asian-African countries and 12.9% from European American countries. There were 14.1% of mothers with up to nine years of education, 56.5% had 10 12 years of education and 29.4% had 13 or more years of education. The vision screening was administered by three trained school nurses using the Snellen chart and the Canon CV-10 machine. Before the administration of the test, the nurse gave an explanation to each class about the two methods to be used. Each child had both tests on the same day. The order in which the classes performed the test was randomly selected and nurses performing the test were randomly allocated to each class. Results of the first test administered were not known to the nurse administering the second test. F o r the Snellen test the child stood five metres (six yards) from the chart. This distance was chosen to be comparable to the one used by the Canon test. Each eye was tested separately. The C a n o n machine uses a broken circle, the Landolt rings, 5 as the test method. The child uses a joystick to indicate in which segment the gap of the circle is located. Both eyes were checked. In this test for monocular vision, there is no need to cover one eye. A synthesized voice guides the examinee through the procedure. Results of the test are automatically printed when the test is over. Children who had a vision of >~6/12 in either eye or two lines difference according to the Snellen test were referred to the specialist. The duration of the screening procedure was measured by a stop-watch for the Snellen test and automatically by the built-in clock in the C a n o n machine. Students from the sixth grade were responsible for measuring the duration of the Snellen test and they also helped the nurse to organize the screening activities. At the end of both procedures, children were asked about the difficulties and their satisfaction with each test. Data were analysed using SPSSX. 6 To test the validity o f the new screening method, the Snellen test was used as a 'gold standard'. The results were dichotomized as 'normal' (a value of 6/6, 6/9) and ' a b n o r m a l ' (a value o f >~6/12). Different cut-off points were selected for the C a n o n machine. The calculation o f sensitivity and specificity was as follows:
Snellen abnormal normal ~>6/12 ~ 6/12 for either eye according to the Snellen test. Five (4.1%) o f the students in the second grade and seven (4.7%) in the fifth grade were already known. The comparison between both screening tests is seen in Tables I and II. The specificity of the test (those identified as normal by the C a n o n test, a m o n g normal children by the Snellen test), increased f r o m 64.4% for a cut-off point o f >~6/10 to 86.5% for a cut-off point o f ~>6/12 or >~6/15 a m o n g children in the second grades, and from 77.2% to 90.6% and 91.3% for the same cut-off points a m o n g fifth graders. Concurrently, between 14-36% of second graders and 9 23% o f fifth graders will be over-referred (false positives). Sensitivity, or those who will be identified as pathological a m o n g children with vision problems by the Table I
Comparison of the automated vision screening test with the Snellen chart as a 'gold standard' Second grade
Fifth grade
Snellen
Snellen
>~6/12 ~~6/12 Canon />6/10
5 7 12
14 90 104
~>6/12 46/9 5 5 l0
12 115 127
Sensitivity
5 12-41.7 (16.5-71.4)
5 10-50.0
(20.1-79.9)
Specificity
oo 1 ~ = 8 6 . 5 (78.1-92.2)
ll5 127-""~u°~
(83.7 94.8)
(c.i. in brackets)
Table II
/> 6/10 >~6/12 ~>6/15
Sensitivity and specificity of the automated vision screening test compared to the Snellen Chart test using three different cut-off points Second grade Sensitivity (c.i.) Specificity (c.i.)
Fifth grade Sensitivity (c.i.) Specificity (c.i.)
66.7 (35.4-88.7) 41.7(16.5 71.4) 25.0(6.7-57.2)
80.0 (44.246.5) 50.0(20.1-79.9) 30.0(8.1-64.6)
64.4 (54.4-73.4) 86.5(78.1-92.2) 86.5(78.1-92.2)
77.2 (68.7-83.9) 90.6(83.7 94.8) 91,3(84.7 95.4)
The scale for the Snellen test is 6/6, 6/9, 6/12, 6/15, 6/30, 6/60. The scale for the Canon test is 6/6, 6/7.5, 6/10, 6/ 12, 6/15, 6/20, 6/30, 6/60.
142
R. Gofin and M. Falk Table III
Reported characteristics of the screening tests by grade Grade
Which test was easier? Snellen Canon Both Don't know Total Which test did you like more? Snellen Canon Both Don't know Total
Second n %
n
48 49 17 2
70 44 26 2
41.4 42.2 14.7 1.7
Fifth % 49.3 31.0 18.3 1.4
116 100.0 142 100.0 17 86 12 -
14.8 12 7 4 . 8 117 10.4 11 3
8.4 81.8 7.7 2.1
115 100.0 143 100.0
Snellen test, was low: 66.7% for a cut-off point of ~>6/10 which decreased to 41.7% and 25.0% for cut-off points of >~6/12 and >~6/15 a m o n g children in the second grade. The sensitivity was higher among fifth graders: 80.0%, 50% and 30%, respectively. Twenty-three children were referred for further assessment. A m o n g the 13 children in the second grade, three obtained glasses (two of them also tested >~6/12 by the Canon test), four were normal (they also screened normal by the Canon test) and no answer was obtained for six of the children (two of them left school and could not be located). A m o n g the latter children, one child did not know how to operate the Canon machine, four were normal and one had a value o f / > 6 / 1 2 . A m o n g the ten children in the fifth grade who were referred, five obtained glasses (four were also screened positive by the C a n o n test); three were normal (two were normal by the Canon test) and for two cases no answer was obtained (one of them left the area); both screened positive by the Canon test. When re-tested 27 (87%) children from the second grade and 25 (83%) from the fifth grade screened normal again. The remaining nine children were screened abnormal the first time and normal the second. Three in each grade screened 6/60 the first time for the first eye examined. The easiness and satisfaction of both procedures are seen in Table Ill. The proportion of second grade children who considered the test easy was similar for both the Snellen and the Canon tests. Older children in the fifth grade reported the Snellen procedure to be easier in higher proportion. However, children in the second grade reported an easier test for the last test performed, whether Snellen or Canon. Both groups of children reported that they preferred the Canon to the Snellen test: 74.8% a m o n g second graders and 81.8% a m o n g fifth graders. The Snellen test took significantly less time than the Canon procedure in both the second and fifth grades (Table IV). Children in the fifth grade did the test more quickly than younger children. The time taken to screen 1,000 students in second grade can be estimated
Comparison of the Automated Vision Screening Test to the Snellen Test Table IV
143
Time (in seconds) taken to administer the screening tests by grade*
Test Snellen Canon
Second grade (n - 118) Mean + s.d.
Fifth grade (n = 144) Mean ± s.d.
47.0 ± 22.4 82.9 zk34.9
39.7 + 17 77.9 ± 28.3
* Student t test P
