Vision Research 108 (2015) 77–84
Contents lists available at ScienceDirect
Vision Research journal homepage: www.elsevier.com/locate/visres
Assessing the utility of visual acuity measures in visual prostheses Avi Caspi a,b, Ari Z. Zivotofsky c,⇑ a
Department of Applied Physics, Jerusalem College of Technology-Lev Academic Center, Jerusalem, Israel Second Sight Medical Products, Inc., Sylmar, CA United States c Brain Science Program, Bar Ilan University, Ramat Gan, Israel b
a r t i c l e
i n f o
Article history: Received 7 October 2014 Received in revised form 14 January 2015 Available online 29 January 2015 Keywords: Visual acuity Retinal prosthesis Pixelized vision Head scanning
a b s t r a c t There are presently several ongoing clinical trials to provide usable sight to profoundly visually impaired patients by means of electrical stimulation of the retina. Some of the blind patients implanted with retinal prosthesis reported un-patterned perception and yet benefit from the device in many activities of daily living, seemingly because they adopt active scanning strategies. The aim of the present work is to evaluate if and under what conditions a measured visual acuity level is truly an indication that the brain perceived a patterned image from the electrical stimulation of the visual prosthesis. Sighted subjects used a pixelized simulator in which they perceived either a low resolution sub-sampling of the original image (‘‘normal mode’’ – patterned vision) or an image that was solely a function of the brightness and size of the original image (‘‘brightness mode’’ – no patterned vision). Results show that subjects were able to adopt a head scanning strategy that enabled acuity beyond the resolution set by a static view of the stimulus. In brightness mode, i.e. without patterned vision, most subjects achieved a measurable acuity level better than the limit set by the geometrical resolution of the entire array but worse than the limit set by the distance between neighboring simulated pixels. In normal mode all subject achieved acuity level that is better than the geometrical resolution of the simulated pixels. Thus, visual acuity levels comparable with the electrodes/pixels resolution implies that the patient perceives an image with spatial patterns. Ó 2015 Elsevier Ltd. All rights reserved.
1. Introduction Therapeutic procedures require an objective method to assess the efficacy of the treatment. Visual acuity tests are considered the principle quantitative measure to assess the efficacy of ophthalmologic treatments and procedures designed to improve or restore vision (Rosenfeld et al., 2006) and to evaluate the costeffectiveness (Kobelt, Lundström, & Stenevi, 2002). Recently, the effort to develop methods to restore vision in totally blind individuals has made important strides, to the extent that a comprehensive review of visual prostheses declared that most of the future obstacles have now been identified (Eiber, Lovell, & Suaning, 2013). The need to assess the objective efficacy and subjective benefit provided by these techniques has raised anew the question of how best to quantify visual functionalities. Results from clinical trials of retinal prosthesis show a great variability in the percept from the electrical stimulation of the degenerated retina.
⇑ Corresponding author at: Gonda Brain Research Center, Bar Ilan University, Ramat Gan, Israel. Fax: +972 3 535 2184. E-mail address: [email protected] (A.Z. Zivotofsky). http://dx.doi.org/10.1016/j.visres.2015.01.006 0042-6989/Ó 2015 Elsevier Ltd. All rights reserved.
In classic Visual Acuity (VA) tests a subject or a patient is required to report the identity of different patterns presented in various sizes. Each size corresponds to a spatial frequency, and the resulting visual acuity is defined by the smallest shape that can be correctly identified by the observer. The most common shapes used for visual acuity tests are letters from the alphabet, such as used in the Snellen chart and in the ETDRS test (Dobson et al., 2009). Non-alphabetic charts and methods were introduced to assess visual acuity for infants and kindergarten children (Ferris et al., 1982). Visual acuity tests based on a closed-set of shapes were also introduced. These tests include the Tumbling E and Landolt C. In those tests, respectively, a letter E or letter C is presented in different orientations and the subject is required to identify the direction of the optotype. In sighted individuals it has been shown that the visual acuity test results reflect the perceptual acuity which is better than the resolution acuity (Heinrich & Bach, 2013). One of the goals of this study was to investigate whether this is true regarding visual prostheses as well, i.e. does the visual acuity score measured in artificial vision reflect an acuity that is superior to the resolution acuity of the sensor. Presently, there are several ongoing clinical trials to ascertain the feasibility of providing usable sight to totally blind patients
78
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
by means of electrical stimulation of the retina. Such therapy is designed for patients who are completely blind due to a progressive retinal degeneration. Sight restoration is done by electrical stimulation of the retina based on the view acquired either by an external video camera (Hornig et al., 2008) or an implanted array of photodiodes (Zrenner et al., 2011). The concept of restoring sensory functionality by means of electrical stimulation is partially driven by the huge success of the cochlear implant that has restored hearing to approximately a quarter of a million individuals worldwide, including numerous children that were born deaf (Papsin & Gordon, 2007). As of today, there is no standardized procedure to quantify the benefit obtained from visual prostheses. It is recommended by the US regulatory agencies that visual acuity is a primary effectiveness endpoint and would be the desired measure. However, it is recognized that standard acuity eye charts are far beyond the ability of today’s prosthesis recipients (Cohen, 2007). Thus, the FDA (2013) in section 7D (Effectiveness Outcomes) in its Guidance for Retinal Prostheses recommends that ‘‘Primary effectiveness endpoints of visual performance should provide quantitative documentation of implanted subjects’ performance in support of device effectiveness. Depending on the patient population and the nature of the underlying condition, the effectiveness endpoints can be selected from the list of assessments below.’’ This list includes the following measures of Visual Function: Low Vision Letter Acuity, Grating Acuity, Spatial Mapping of Stimulated Visual Phosphene Fields, Form Vision Assessment, Assessments of Functional Vision and Patient Reported Outcomes, Orientation and Mobility, Activities of Daily Living, and Patient Reported Outcomes questionnaires. Indeed, outcomes, other than visual acuity are being published as outcome measures for visual prostheses (e.g. Kotecha et al., 2014; Nau et al., 2014). In most European countries and in the United States, a legally blind person is defined as someone who has 1/10th of the normal visual acuity, that is, when a person cannot identify the largest letter on the Snellen chart. Current vision tests that evaluate patients with acuity worse than this acuity, i.e. worse than 6/60 (20/200), are limited and not standardized. There are limited quantifiable visual tests aimed at visual levels between total blindness, i.e., no light perception, and legal blindness. Often, for patients in this range, termed ultra-low vision and the range for all current artificial vision devices, clinicians use methods such as light perception with projection and counting fingers. An effort has been made to quantify VA in patients with severe visual impairment who would normally be evaluated with finger counting and found they could reproducibly quantify VA (Lange et al., 2009; Schulze-Bonsel et al., 2006). Others have noted that within the population of low visual functioning there is poor agreement between the Snellen and ETDRS charts. Often in clinical practice Snellen charts are used while in clinical trials ETDRS charts are utilized (Falkenstein et al., 2008). Recently, Bailey et al. (2012) suggested using The Berkeley Rudimentary Vision Test for low vision visual acuity testing. It consists of three pairs of hinged cards that test using single tumbling E optotypes, various grating acuity targets, and white field projection and black white discrimination. This test is commercially available (e.g. http://precision-vision.com/). Bach et al. (2010) have recently developed a new simple test battery to provide a basic quantitative assessment of visual function in the very-low-vision range. This battery of tests has also been used to evaluate tactile vision substitution, for example tactile stimulation of the tongue (Nau, Bach, & Fisher, 2013). The ability to quantify visual acuity for severe low vision will be of a great use in assessing the results of a variety of modern therapies aimed at the severely impaired patients. Can the extended range of these modified visual acuity tests be used to quantify the vision provided by a visual implant? There is no doubt that the vision provided by the current visual prostheses
is different from that of normal human vision. Nevertheless, even crude and artificial vision yields a valuable benefit to blind patients that do not have an alternative treatment (Ahuja et al., 2011). The traditional visual acuity measurements assume that the patient has a spatial map of the image, i.e., can perceive patterns or shapes. Preliminary outcomes of retinal prostheses’ clinical trials have shown that some of the participants cannot identify patterns or shapes. However, participants do benefit by their newly acquired ability to locate objects and detect motion in their daily activities. Yet there is no accepted method to quantify this acquired vision (Cohen, 2007). Due to the different pathologies of diseases that cause blindness, the outcome of a visual prosthesis is patient specific and thus, while some patients are able to identify patterns and can score on the extended visual acuity test, other patients can only locate objects or detect motion (Caspi et al., 2009; Humayun et al., 2012; Stingl et al., 2013). In order to gain a better understanding of the potential benefits of low resolution visual prostheses and to assess different image processing algorithms, visual prosthesis simulators are used. Generally, in a visual prosthesis simulator, also known as pixelized vision simulator, a real-time, low-resolution image of the view is presented on LCD goggles to a normally sighted user. The image of the scene is captured by a head mounted camera, digitalized by a computer, and a sub-sampled low resolution (‘‘pixelized’’) image is presented on a commercial eyewear video display (Fig. 1). A variety of tasks have been evaluated using pixelized vision simulator. Thompson et al. (2003) investigated the minimum requirements for face recognition and Fornos et al. (2005) used a visual prosthesis simulator to compare the scanning benefit and shape of individual pixels, square vs. Gaussian, in enabling reading. Hallum et al. (2005) explored the effect of pixelized vision on various eye movements, i.e. smooth pursuit, saccades, and fixation. Wang, Yang, and Dagnelie (2008a) investigated the effect of retinal location of the projected pixelized image on smooth pursuit initiation and stability. Dagnelie et al. (2007) and Wang, Yang, and Dagnelie (2008b) assessed virtual maze navigation and real mobility performance with simulated prosthetic vision. Parikh et al. (2013) compared various computer algorithms, including saliency-based cueing algorithms, using a visual prosthesis simulator. Clinical trials of visual prosthetics showed that some patients cannot perceive shapes. Published reports from 30 patients implanted with the Argus II prosthesis (Humayun et al., 2012) showed that only 23% can discriminate the orientation of a grating while 57% could discriminate motion and 96% of the patients were able to localize objects. Results from the 8 patients implanted with the Alpha IMS clinical trial (Stingl et al., 2013) showed that only 2/8 were able to score on the Landolt C test, 5/8 discriminate
Fig. 1. An image of the pixelized vision simulator which consists of a USB camera and miniature LCD monitors mounted on goggles.
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
gratings and identify motion, and 7/8 were able to performed a localization test. Although the testing methodologies were different in the two clinical trials, both reports hint that not all implanted patients see a pixelized distinct pattern. Therefore for a pixelized visual simulator to be a useful tool and accurately mimic retinal prostheses, they too must take into account the unpatterned perceived image. Herein, we used a pixelized visual simulator with both patterned and unpatterned images in order to evaluate if and under what conditions a measured visual acuity level is truly an indication that the visual prosthesis provides the patient with a patterned image. We compared the results of Landolt C visual acuity tests in two extreme conditions. In the first condition, visual acuity was measured using a classical pixelized image in which each pixel represented the brightness level in the receptive field of the pixel. This mapping reduces the resolution but preserves the stimulus pattern. In the second condition, we used a novel mapping which encoded only the total brightness in the field of view of the stimulus array. This mapping algorithm represents an unpatterned stimulus, hypothesized to occur in some patients implanted with retinal prostheses. In this mode there is no geometric resolution present in a static image and it would be impossible to derive any spatial information from such an image. Based solely on (non-existent) pattered information, patients with such vision should be unable to exhibit a measured visual acuity.
2. Methods 2.1. Pixelized vision simulator A Small, commercial USB WebCam (PK-836MJ, A4Tech, China) was attached to glasses that contained binocular Liquid Crystal Displays (AV230, Vuzix, USA) in order to provide an image with a resolution of 8 8 pixels. The WebCam was set to a resolution of 160 120 and images were sampled in real time by a Matlab application using the freeware vcapg2 function. The pixelized simulator was able to run in two modes, a Normal mode and a Brightness mode. In Normal Mode, a sub-sampled, 8 8, pixelized image with spatial information was created from the raw image by averaging the brightness in the corresponding field-ofview. Each pixel in the pixelized image was created by averaging the matching 15 15 pixels out of the 160 120 pixels in the raw image from the camera digitizing to a dynamic range of 5 levels. In this mode the resolution is set by the distance between neighboring pixels, which was 1.75°. This is equivalent to a stimulus of 2.0 log MAR (=log10 (60 1.75)). In Brightness Mode, the 8 8 pixelized image had the same brightness and size of the original image without any spatial patterns. It is worthwhile to mention that the perceived visual acuity is most likely better than the above resolution acuity (Heinrich & Bach, 2013). However, resolution acuity is usually given as a reference in order to evaluate the simulated implanted visual sensor. In order to create the unpatterned Brightness mode image, first a normal mode pixelized image was created and then the pixels were placed in the center in order to create a single structure without pattern. The unpatterned brightness mode image has the same overall brightness as the pixelized normal mode but without the pattern. The pixels were rearranged in the center of the image based on their brightness, the brightest pixels in the center. This creates a continuous, smooth percept. In this mode the entire simulated image is viewed as a ‘‘one-pixel’’ system and the resolution is set by the field of view. The total captured field-of-view of the 8 8 pixels is 14° which is 2.9 log MAR, which is 8 times larger than in the normal mode. Examples of each mode are shown in Figs. 2 and 3.
79
2.2. Visual acuity test Visual acuity was measured using the adaptive Freiburg Visual Acuity & Contrast Test (FrACT; Bach, 1996). The stimuli were Landolt-Cs in which the letter ‘‘C’’ is presented in one of 4 orientations and the subject needs to identify the side in which the gap appears. This is a freeware application that can be downloaded at: http:// www.michaelbach.de. The program was set to the 4 Alternative Force Choices (4AFC) task. In the FrACT program the visual acuity in each trial was set according to a maximum likelihood based on responses for all previous trials. Every sixth trial was an ‘‘easy’’ trial, in order to maintain the subject’s motivation. Maximum presentation time for each trial was 60 s, but concluded sooner if the subject reported his/her response earlier. 2.3. Subjects Six sighted volunteers, students aged 20–30 (3 female; average age 24), with normal or corrected to normal vision, participated in the experiment. The research was approved by the Ethics Committee of Bar Ilan University and was performed in accordance with the Declaration of Helsinki. Prior to the experiment each subject gave informed consent. Some of the subjects received course credit for participating in the study. All subject were naïve as to the aim of the experiment, but were subsequently informed about the goals and motivation of the study. 2.4. Procedure Visual acuities and reaction times of the subjects using the pixelized vision simulator in normal and brightness modes were measured using FrACT Landolt-C visual acuity test. Subjects sat in a darkened room in front of a 19 in. monitor at a 36 cm distance while wearing the Pixelized Vision Simulator. Initially the simulator was set to normal mode in which a low resolution, 8 8, subsampled image of the target was presented on the LCD of the glasses (snapshot in Fig. 2a) and subjects were accustomed to vision via the simulator by presenting gratings in various orientations. This was followed by the assessment of the visual acuity using the FrACT set to 30 trials at the normal mode of the simulator. The simulator was then switched to brightness mode in which a low resolution 8 8, pixelized image with the same brightness and size of the original image but without any spatial patterns was presented (see Fig. 2b) and the procedure for adapting and then assessing visual acuity was repeated. In both modes the subjects were permitted to move their head in order to scan the target by steering the line of sight of the camera. However, there were no explicit instructions regarding head movements and any such movements were due to the subject having ‘‘learned’’ that it can be of assistance. 2.5. Statistical analysis The validity of the FrACT Landolt-C visual acuity test was verified post hoc by analyzing the ensemble of the trials in the bracket of plus–minus 0.1 log MAR around the reported visual acuity of the adaptive algorithm. Using the cumulative binomial distribution we evaluated the probability that the results in this bracket were not driven by visual information, i.e. guessing. The binomial distribution was evaluated taking into account that on each trial the probability of reporting correctly by guessing is 0.25. Results were considered significant if the probability to score by guessing in the bracket of plus–minus 0.1 log MAR around the reported visual acuity of the adaptive algorithm was less than 0.01.
80
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
Fig. 2. Snapshots of the pixelized vision simulator in normal and brightness modes with a gap size that is equal to 1/2 the size of an electrode. The left column shows images of the stimulus as captured by the USB camera. The rectangle represents the field of view of the simulated prosthesis which is identical for all cases. The middle column shows the pixelized image in the normal mode. The Right column shows the pixelized image in brightness mode. Each row shows a different snapshot during scanning. It is noticeable that for the sub-pixel gap the optotype direction is not visible in all snapshots and hyper-acuity is achieved by scanning, which adds an additional dimension, that of time.
3. Results Visual acuities for the 6 subjects in the two modes with the Landolt-C test are shown in Fig. 4. The left column of Fig. 4 presents the results in normal mode while the result of the brightness mode is given in the right column. These acuities are summarized in Table 1. In the brightness mode one subject was not able to reach a level significantly greater than chance. All other results were significant with a probability greater than 0.99. In normal mode, visual acuity for all six subjects was at least 0.2 log MAR better than the geometrical resolution of the simulated pixelized vision (2.0 log MAR). The addition of a time dimension utilized by a head scanning strategy enabled them to achieve this acuity level. In brightness mode in which there is no patterned vision and performance is theoretically limited by the field of view
of the system, 4 out of the 6 subjects similarly adapted a head scanning strategy that enabled them to achieve an acuity level that is at least 0.5 log MAR better than the limit set by the field of view of the system (2.9 log MAR). Perhaps, with training subjects might perform better. Of the other two subjects, one achieved an acuity level equal to the geometrical limit and the other did not succeed in developing a scanning strategy and thus did not score above chance on any acuity level when the simulated vision did not contain a pattern. The response times of the subjects are given in Table 1. In brightness mode it took the subjects up to 3 times longer than in Normal mode to report their answer, although in Normal mode the response times were also longer than the time usually taken to respond to a visual acuity test. These times were due to the time required to scan the image in order to identify the location of the
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
81
Fig. 3. Snapshots of the pixelized vision simulator in normal and brightness modes with a gap size that is equal to half the field of view of the array. The left column shows images of the stimulus as captured by the USB camera. The rectangle represents the field of view of the simulated prosthesis which is identical for all cases. The middle column shows the pixelized image in the normal mode. The Right column shows the pixelized image in brightness mode. Each row shows a different snapshot during scanning. It is noticeable that in brightness mode it is impossible to detect the optotype from a single sample and visual functionality is achieved only by and because of the temporal scanning.
gap based on changes of the overall brightness in the field-of-view of the imager. 4. Discussion Herein we showed that in a manner similar to that observed with sighted individuals (Heinrich & Bach, 2013) simulated pixelized vision yields visual acuities better than the resolution acuity sets by the sensor. However, the time required for obtaining that acuity via a head scanning strategy is on the order of 10 s. Moreover, we showed that the measured perceptual acuity is better even in the absence of any resolution acuity as indicated by the Brightness Mode which required scanning time on the order of minutes. The results from the two modes offer insight into two phenomena that are observed in blind patients implanted with retinal
prostheses (Humayun et al., 2012). During Normal Mode we showed that while using a pixelized image simulator subjects can score acuity levels that are significantly better than would be expected based on the geometrical resolution of the system. In a previous study, a patient implanted with an epi-retinal prosthesis was able to score an acuity level that was significantly better than the limit set by the distance between neighboring electrodes (Humayun et al., 2012; Mohand-Said et al., 2011). Most likely, in that case the brain integrated information over time and used head scanning to achieve better than expected resolution, as seen in our subjects. This is similar to adding jitter to the image, a technique that is known to enhance visual performance when spatial resolution is limited (Watson et al., 2012). The benefit of head scanning in improving apparent visual acuity in pixeled vision system is long recognized. Already in 1992 Cha, Horch, and Normann (1992) anticipated the development of
82
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
Fig. 4. Visual acuity data of the 6 subjects using the pixelized image simulator in normal and brightness modes. Charts show the acuity level as a function of trial number. Left column panels show the results of runs in normal mode. Right column panels are the results of runs in normal mode. Filled circles represent a correct response, while a circle with an ‘‘X’’ represents a trial with an incorrect response. The estimated visual acuity, percent correct, is 62.5%, a mid-point between 100% and chance level (25%). Estimated visual acuity is indicated by the dashed line and the number to the right of the chart. In order to keep the subjects motivated, every sixth trial was an easy trial.
Table 1 Summary of the data. Table provides measured visual acuity and mean response time for each subject for both modes (Normal and Brightness). Subject ID
MSX-001 ASK-002 ASX-003 SHA-004 RBS-005 ADK-006
Gender
Male Female Male Female Male Female
Normal mode
Brightness mode
Visual acuity (log MAR)
Mean response time (sec)
Visual acuity (log MAR)
Mean response time (sec)
1.6 1.6 1.8 1.8 1.6 1.7
9.5 8.8 5.4 15.0 11.5 8.3
2.2 2.6 2.4 2.3 2.4 N.A.
21.0 39.0 16.8 41.8 28.1 25.5
visual prostheses and performed psychophysical experiments using a phosphene simulator on six subjects. They found that while neither eye movements nor smooth movements of the target affected visual acuity, ‘‘voluntary head movements did help our
subjects with the low pixel density masks. For example, our subjects showed a sudden improvement in performance right after they were encouraged to use head movements’’ (page 446). They postulate that this improvement may have been due to either
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
temporal integration while the object was scanned or the use of head movements to find the best viewing position. It should be noted that their experiments involved multiple sessions over a 3 month period. In contrast, we showed that subjects can adapt head scanning without extensive training, either to improve visual acuity in normal mode or to overcome the unpatterned image in brightness mode. In addition, all of their stimuli were pixelized, i.e. no equivalent of our brightness mode. Finally, we gave no instructions that would have encouraged head scanned; it seems that scanning is a natural process in the visual system that subjects employed on their own. More recently, Chen et al. (2006) built on Cha et al.’s findings and investigated what properties of head scanning optimize the scanning’s benefit. Using a simulation of prosthetic vision they had subjects perform ten sessions in which the phosphene lattice offered a theoretical equivalent of 2.0 log MAR. All subjects had a measured acuity greater than that thanks to head scanning. Furthermore, circular scanning seems to afford the greatest benefit, as does higher velocity scanning. As in our study, they offered no specific guidance. However, they too had no equivalent of our brightness mode as all stimuli were pixelized images that preserved the pattern of the image, which is not the case for many of the blind patients implanted with prostheses. Thus, our findings in ‘‘Normal Mode’’ are not totally unexpected. They extend previous findings regarding patterned vision in normally sighted individuals to a model of what retinal prosthetic patients with patterned vision might experience and introduce the importance of scanning to the literature dealing with the burgeoning field of visual prostheses. In Brightness Mode, in which the simulator delivered an image based only on intensity without any pattern information, subjects were able to score on the visual acuity test, but with a score that is still significantly worse (average of 2.2 log MAR) average than the spatial resolution set by the distance between pixels (2.0 log MAR), the equivalent of electrodes in prosthetic systems. Nonetheless, this is far better than when the system is looked upon as a ‘‘one-pixel’’ system, as it is in brightness mode. Note, that the brightness mode is not truly one pixel, because the width can change and it is not uniform. These results indicate that the subjects probably ‘‘saw’’ something and scored well above chance, which is better than would be expected from mere brightness information. This was most likely accomplished using a tedious and time consuming acquisition of information via a systematic scanning pattern. Support for this comes from the response time information. While a sighted subject in native vision responds in less than a second, the response using Normal Mode was on the order of 10 s (9.8 ± 3 s) and during Brightness Mode 2–3 times longer (28.7 ± 9.9 s). These findings regarding the brightness mode, i.e. unpatterned vision, are entirely novel and are very important in understanding the measured visual acuity and the observed visual functionality of individuals with retinal prostheses. Extended reaction time was found in a study of facial expression recognition using simulated prosthetic vision (van Rheede, Kennard, & Hicks, 2010). They employed different trade-offs between field of view and resolution and while there was a significant difference in the acuity that corresponded with the resolution, the reaction time for the facial recognition task was the same for all resolutions and field of view (approximately 4 s). We suggest that just as in our case, their extended reaction times are the result of strategies employed by the subjects of scanning in order to increase the usable information. In general, when clinical visual acuity (VA) is measured, time is not factored in. Most optometrists simply run through a standard test without giving a thought to how long the patient looks at each item. If a patient takes an inordinate amount of time to struggle
83
through a line of letters on an acuity chart, the clinician will usually not record that line as the patient’s visual acuity. On the other hand, with low vision patients, a clinician may opt to record either the acuity performed with ease or the one that the patient struggled with but was able to read with extra time. This second acuity number two reveals the potential of the low vision patient. In other words, normally sighted and visually impaired persons cannot be assessed in the same way. In testing anyone other than a ‘‘perfect’’ subject, such as a patient with artificial vision, time is indeed an important factor. Sometimes, for example, with a patient with albinism, an optometrist may subconsciously (or consciously) give them extra time, but that is then not taken into account in the final acuity measure. Clark and Clark (2013) showed that giving more time vastly improved the grating acuity for two cats. Yang et al. (2005) investigated the effect of limiting gazing time to 550 ms during an acuity test. They found that indeed for healthy subjects there was no difference in measured acuity between notime-restricted acuity and time-restricted acuity. However, for patients with Infantile Nystagmus Syndrome there was a significant improvement in reported acuity when they are given more time to examine each optotype. It is clear that individuals with compromised vision will compensate and develop time-consuming strategies to improve their acuity. In our case with a pixelized vision simulator, in both modes the acuity is severely limited, yet subjects developed strategies such that when given sufficient time would enable them to achieve a relatively high reportable acuity. This is despite the fact that they do not necessarily have that level of form vision. It is important that reports of visual acuities measured on blind with retinal prosthesis will report the response time or if time was limited. For example, in the Argus II clinical trials the reported visual acuities were measured with a time limit of five seconds (Humayun et al., 2012). Imposing a short time limit may increase the chances of measuring a ‘‘true’’ visual acuity. Or alternatively, it is important to report the actual reaction times together with the measured acuity. In conclusion, measurements of visual acuity are not necessarily an indication that the visual prosthesis provides an image with spatial patterns. Nevertheless, visual acuity levels at or better than the sub-pixel resolutions do seem to imply that the patient perceives an image with spatial patterns. Our study results show that there is not necessarily a one-to-one correspondence between visual acuity measurements and the exact spatial pattern produced by the retinal prosthesis device. We note that there is a difference between resolution acuity and recognition acuity. Vision with retinal prostheses – at least with those which can utilize scanning movements of either the eye or head – shows the known phenomenon that recognition acuity can be considerably better than expected from geometrical resolution acuity, and that this is similar to findings with Landolt C-ring based methods of VA assessment in normal vision. References Ahuja, A. K., Dorn, J. D., Caspi, A., McMahon, M. J., Dagnelie, G., Dacruz, L., et al. (2011). Argus II Study Group. Blind subjects implanted with the Argus II retinal prosthesis are able to improve performance in a spatial-motor task. British Journal of Ophthalmology, 95, 539–543. Bach, M. (1996). The ‘‘Freiburg Visual Acuity Test’’ – Automatic measurement of visual acuity. Optometry and Vision Science, 73, 49–53. Bach, M., Wilke, M., Wilhelm, B., Zrenner, E., & Wilke, R. (2010). Basic quantitative assessment of visual performance in patients with very low vision. Investigative Ophthalmology & Visual Science, 51, 1255–1260. Bailey, I. L., Jackson, A. J., Minto, H., Greer, R. B., & Chu, M. A. (2012). The Berkeley rudimentary vision test. Optometry and Vision Science, 89(9), 1257–1264. Caspi, A., Dorn, J. D., McClure, K. H., Humayun, M. S., Greenberg, R. J., & McMahon, M. J. (2009). Feasibility study of a retinal prosthesis: Spatial vision with a 16electrode implant. Archives of Ophthalmology, 127, 398–401.
84
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
Cha, K., Horch, K., & Normann, R. A. (1992). Simulation of a phosphene-based visual field: Visual acuity in a pixelized vision system. Annals of Biomedical Engineering, 20(4), 439–449. Chen, S. C., Hallum, L. E., Suaning, G. J., & Lovell, N. H. (2006). Psychophysics of prosthetic vision: I. Visual scanning and visual acuity. Conference Proceedings on IEEE Engineering in Medicine and Biology Society, 1, 4400–4403. Clark, D. L., & Clark, R. A. (2013). The effects of time, luminance, and high contrast targets: Revisiting grating acuity in the domestic cat. Experimental Eye Research, 116, 75–78. Cohen, E. (2007). Safety and effectiveness considerations for clinical studies of visual prosthetic devices. Journal of Neural Engineering, 4, S124–S129. Dagnelie, G., Keane, P., Narla, V., Yang, L. C., Weiland, J., & Humayun, M. (2007). Real and virtual mobility performance in simulated prosthetic vision. Journal of Neural Engineering, 4(1), S92–S101. Dobson, V., Clifford-Donaldson, C. E., Miller, J. M., Garvey, K. A., & Harvey, E. M. (2009). A comparison of Lea symbol vs ETDRS letter distance visual acuity in a population of young children with a high prevalence of astigmatism. JAAPOS, 13, 253–257. Eiber, C. D., Lovell, N. H., & Suaning, G. J. (2013). Attaining higher resolution visual prosthetics: A review of the factors and limitations. Journal of Neural Engineering, 10(1). Falkenstein, I. A., Cochran, D. E., Azen, S. P., Dustin, L., Tammewar, A. M., Kozak, I., et al. (2008). Comparison of visual acuity in macular degeneration patients measured with snellen and early treatment diabetic retinopathy study charts. Ophthalmology, 115, 319–323. FDA Guidance for Industry and FDA Staff – Investigational Device Exemption (IDE) Guidance for Retinal Prostheses, 2013. Ferris, F. L., Kassoff, A., Bresnick, G. H., & Bailey, I. (1982). New visual acuity charts for clinical research. American Journal of Ophthalmology, 94, 91–96. Fornos, A. P., Sommerhalder, J., Rappaz, B., Safran, A. B., & Pelizzone, M. (2005). Simulation of artificial vision, III: Do the spatial or temporal characteristics of stimulus pixelization really matter? Investigative Ophthalmology & Visual Science, 46, 3906–3912. Hallum, L. E., Suaning, G. J., Taubman, D. S., & Lovell, N. H. (2005). Simulated prosthetic visual fixation, saccade, and smooth pursuit. Vision Research, 45, 775–788. Heinrich, S. P., & Bach, M. (2013). Resolution acuity versus recognition acuity with Landolt-style optotypes. Graefes Archive for Clinical and Experimental Ophthalmology, 251, 2235–2241. Hornig, R., Zehnder, T., Velikay-Parel, M., Laube, T., Feucht, M., & Richard, G. (2008). The IMI retinal implant system in artificial sight. Biological and Medical Physics, Biomedical Engineering, 111–128. Humayun, M. S., Dorn, J. D., da Cruz, L., Dagnelie, G., Sahel, J. A., Stanga, P. E., et al. (2012). Argus II Study Group interim results from the international trial of second sight’s visual prosthesis. Ophthalmology, 119, 779–788. Kobelt, G., Lundström, M., & Stenevi, U. (2002). Cost-effectiveness of cataract surgery. Method to assess cost-effectiveness using registry data. Journal of Cataract and Refractive Surgery, 28, 1742–1749. Kotecha, A., Zhong, J., Stewart, D., & da Cruz, L. (2014). The Argus II prosthesis facilitates reaching and grasping tasks: A case series. BMC Ophthalmology, 23(14), 71.
Lange, C., Feltgen, N., Junker, B., Schulze-Bonsel, K., & Bach, M. (2009). Resolving the clinical acuity categories ‘‘hand motion’’ and ‘‘counting fingers’’ using the Freiburg Visual Acuity Test (FrACT). Graefes Archive for Clinical and Experimental Ophthalmology, 247, 137–142. Mohand-Said, S., Caspi, A., Merlini, F., et al. (2011). Comparison of Etdrs, Landolt C, and grating visual acuity tests between sighted volunteers using a pixelized image simulator and blind subjects implanted with the ArgusTm Ii retinal prosthesis. Investigative Ophthalmology & Visual Science, 52. E-Abstract 4931. Nau, A., Bach, M., & Fisher, C. (2013). Clinical tests of ultra-low vision used to evaluate rudimentary visual perceptions enabled by the brainport vision device. Translational Vision Science & Technology, 2(3), 1. Nau, A. C., Pintar, C., Fisher, C., Jeong, J. H., & Jeong, K. (2014). A standardized obstacle course for assessment of visual function in ultra low vision and artificial vision. Journal of Visualized Experiments, 84. Papsin, B. C., & Gordon, K. A. (2007). Cochlear implants for children with severe-toprofound hearing loss. New England Journal of Medicine, 357, 2380–2387. Parikh, N., Itti, L., Humayun, M., & Weiland, J. (2013). Performance of visually guided tasks using simulated prosthetic vision and saliency-based cues. Journal of Neural Engineering, 10(2). Rosenfeld, P. J., Brown, D. M., Heier, J. S., Boyer, D. S., Kaiser, P. K., Chung, C. Y., et al. (2006). MARINA Study Group Ranibizumab for neovascular age-related macular degeneration. New England Journal of Medicine, 355, 1419–1431. Schulze-Bonsel, K., Feltgen, N., Burau, H., Hansen, L., & Bach, M. (2006). Visual acuities ‘‘hand motion’’ and ‘‘counting fingers’’ can be quantified with the Freiburg Visual Acuity Test. Investigative Ophthalmology & Visual Science, 47, 1236–1240. Stingl, K., Bartz-Schmidt, K. U., Besch, D., Braun, A., Bruckmann, A., Gekeler, F., et al. (2013). Artificial vision with wirelessly powered subretinal electronic implant alpha-IMS. Proceedings of the Royal Society B: Biological Sciences, 280(1757). Thompson, R. W., Barnett, G. D., Humayun, M. S., & Dagnelie, G. (2003). Facial recognition using simulated prosthetic pixelized vision. Investigative Ophthalmology & Visual Science, 44, 5035–5042. van Rheede, J. J., Kennard, C., & Hicks, S. L. (2010). Simulating prosthetic vision: Optimizing the information content of a limited visual display. Journal of Vision, 10(14). Wang, L., Yang, L. C., & Dagnelie, G. (2008a). Initiation and stability of pursuit eye movements in simulated retinal prosthesis at different implant locations. Investigative Ophthalmology & Visual Science, 49, 3933–3939. Wang, L., Yang, L. C., & Dagnelie, G. (2008b). Virtual way finding using simulated prosthetic vision in Gaze-locked viewing. Optometry and Vision Science, 85, 1057–1063. Watson, L. M., Strang, N. C., Scobie, F., Love, G. D., Seidel, D., & Manahilov, V. (2012). Image jitter enhances visual performance when spatial resolution is impaired. Investigative Ophthalmology & Visual Science, 53, 6004–6010. Yang, D., Hertle, R. W., Hill, V. M., & Stevens, D. J. (2005). Gaze-dependent and timerestricted visual acuity measures in patients with Infantile Nystagmus Syndrome (INS). American Journal of Ophthalmology, 139, 716–718. Zrenner, E., Bartz-Schmidt, K. U., Benav, H., Besch, D., Bruckmann, A., Gabel, V. P., et al. (2011). Subretinal electronic chips allow blind patients to read letters and combine them to words. Proceedings of the Royal Society B: Biological Sciences, 278(1711), 1489–1497.
Contents lists available at ScienceDirect
Vision Research journal homepage: www.elsevier.com/locate/visres
Assessing the utility of visual acuity measures in visual prostheses Avi Caspi a,b, Ari Z. Zivotofsky c,⇑ a
Department of Applied Physics, Jerusalem College of Technology-Lev Academic Center, Jerusalem, Israel Second Sight Medical Products, Inc., Sylmar, CA United States c Brain Science Program, Bar Ilan University, Ramat Gan, Israel b
a r t i c l e
i n f o
Article history: Received 7 October 2014 Received in revised form 14 January 2015 Available online 29 January 2015 Keywords: Visual acuity Retinal prosthesis Pixelized vision Head scanning
a b s t r a c t There are presently several ongoing clinical trials to provide usable sight to profoundly visually impaired patients by means of electrical stimulation of the retina. Some of the blind patients implanted with retinal prosthesis reported un-patterned perception and yet benefit from the device in many activities of daily living, seemingly because they adopt active scanning strategies. The aim of the present work is to evaluate if and under what conditions a measured visual acuity level is truly an indication that the brain perceived a patterned image from the electrical stimulation of the visual prosthesis. Sighted subjects used a pixelized simulator in which they perceived either a low resolution sub-sampling of the original image (‘‘normal mode’’ – patterned vision) or an image that was solely a function of the brightness and size of the original image (‘‘brightness mode’’ – no patterned vision). Results show that subjects were able to adopt a head scanning strategy that enabled acuity beyond the resolution set by a static view of the stimulus. In brightness mode, i.e. without patterned vision, most subjects achieved a measurable acuity level better than the limit set by the geometrical resolution of the entire array but worse than the limit set by the distance between neighboring simulated pixels. In normal mode all subject achieved acuity level that is better than the geometrical resolution of the simulated pixels. Thus, visual acuity levels comparable with the electrodes/pixels resolution implies that the patient perceives an image with spatial patterns. Ó 2015 Elsevier Ltd. All rights reserved.
1. Introduction Therapeutic procedures require an objective method to assess the efficacy of the treatment. Visual acuity tests are considered the principle quantitative measure to assess the efficacy of ophthalmologic treatments and procedures designed to improve or restore vision (Rosenfeld et al., 2006) and to evaluate the costeffectiveness (Kobelt, Lundström, & Stenevi, 2002). Recently, the effort to develop methods to restore vision in totally blind individuals has made important strides, to the extent that a comprehensive review of visual prostheses declared that most of the future obstacles have now been identified (Eiber, Lovell, & Suaning, 2013). The need to assess the objective efficacy and subjective benefit provided by these techniques has raised anew the question of how best to quantify visual functionalities. Results from clinical trials of retinal prosthesis show a great variability in the percept from the electrical stimulation of the degenerated retina.
⇑ Corresponding author at: Gonda Brain Research Center, Bar Ilan University, Ramat Gan, Israel. Fax: +972 3 535 2184. E-mail address: [email protected] (A.Z. Zivotofsky). http://dx.doi.org/10.1016/j.visres.2015.01.006 0042-6989/Ó 2015 Elsevier Ltd. All rights reserved.
In classic Visual Acuity (VA) tests a subject or a patient is required to report the identity of different patterns presented in various sizes. Each size corresponds to a spatial frequency, and the resulting visual acuity is defined by the smallest shape that can be correctly identified by the observer. The most common shapes used for visual acuity tests are letters from the alphabet, such as used in the Snellen chart and in the ETDRS test (Dobson et al., 2009). Non-alphabetic charts and methods were introduced to assess visual acuity for infants and kindergarten children (Ferris et al., 1982). Visual acuity tests based on a closed-set of shapes were also introduced. These tests include the Tumbling E and Landolt C. In those tests, respectively, a letter E or letter C is presented in different orientations and the subject is required to identify the direction of the optotype. In sighted individuals it has been shown that the visual acuity test results reflect the perceptual acuity which is better than the resolution acuity (Heinrich & Bach, 2013). One of the goals of this study was to investigate whether this is true regarding visual prostheses as well, i.e. does the visual acuity score measured in artificial vision reflect an acuity that is superior to the resolution acuity of the sensor. Presently, there are several ongoing clinical trials to ascertain the feasibility of providing usable sight to totally blind patients
78
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
by means of electrical stimulation of the retina. Such therapy is designed for patients who are completely blind due to a progressive retinal degeneration. Sight restoration is done by electrical stimulation of the retina based on the view acquired either by an external video camera (Hornig et al., 2008) or an implanted array of photodiodes (Zrenner et al., 2011). The concept of restoring sensory functionality by means of electrical stimulation is partially driven by the huge success of the cochlear implant that has restored hearing to approximately a quarter of a million individuals worldwide, including numerous children that were born deaf (Papsin & Gordon, 2007). As of today, there is no standardized procedure to quantify the benefit obtained from visual prostheses. It is recommended by the US regulatory agencies that visual acuity is a primary effectiveness endpoint and would be the desired measure. However, it is recognized that standard acuity eye charts are far beyond the ability of today’s prosthesis recipients (Cohen, 2007). Thus, the FDA (2013) in section 7D (Effectiveness Outcomes) in its Guidance for Retinal Prostheses recommends that ‘‘Primary effectiveness endpoints of visual performance should provide quantitative documentation of implanted subjects’ performance in support of device effectiveness. Depending on the patient population and the nature of the underlying condition, the effectiveness endpoints can be selected from the list of assessments below.’’ This list includes the following measures of Visual Function: Low Vision Letter Acuity, Grating Acuity, Spatial Mapping of Stimulated Visual Phosphene Fields, Form Vision Assessment, Assessments of Functional Vision and Patient Reported Outcomes, Orientation and Mobility, Activities of Daily Living, and Patient Reported Outcomes questionnaires. Indeed, outcomes, other than visual acuity are being published as outcome measures for visual prostheses (e.g. Kotecha et al., 2014; Nau et al., 2014). In most European countries and in the United States, a legally blind person is defined as someone who has 1/10th of the normal visual acuity, that is, when a person cannot identify the largest letter on the Snellen chart. Current vision tests that evaluate patients with acuity worse than this acuity, i.e. worse than 6/60 (20/200), are limited and not standardized. There are limited quantifiable visual tests aimed at visual levels between total blindness, i.e., no light perception, and legal blindness. Often, for patients in this range, termed ultra-low vision and the range for all current artificial vision devices, clinicians use methods such as light perception with projection and counting fingers. An effort has been made to quantify VA in patients with severe visual impairment who would normally be evaluated with finger counting and found they could reproducibly quantify VA (Lange et al., 2009; Schulze-Bonsel et al., 2006). Others have noted that within the population of low visual functioning there is poor agreement between the Snellen and ETDRS charts. Often in clinical practice Snellen charts are used while in clinical trials ETDRS charts are utilized (Falkenstein et al., 2008). Recently, Bailey et al. (2012) suggested using The Berkeley Rudimentary Vision Test for low vision visual acuity testing. It consists of three pairs of hinged cards that test using single tumbling E optotypes, various grating acuity targets, and white field projection and black white discrimination. This test is commercially available (e.g. http://precision-vision.com/). Bach et al. (2010) have recently developed a new simple test battery to provide a basic quantitative assessment of visual function in the very-low-vision range. This battery of tests has also been used to evaluate tactile vision substitution, for example tactile stimulation of the tongue (Nau, Bach, & Fisher, 2013). The ability to quantify visual acuity for severe low vision will be of a great use in assessing the results of a variety of modern therapies aimed at the severely impaired patients. Can the extended range of these modified visual acuity tests be used to quantify the vision provided by a visual implant? There is no doubt that the vision provided by the current visual prostheses
is different from that of normal human vision. Nevertheless, even crude and artificial vision yields a valuable benefit to blind patients that do not have an alternative treatment (Ahuja et al., 2011). The traditional visual acuity measurements assume that the patient has a spatial map of the image, i.e., can perceive patterns or shapes. Preliminary outcomes of retinal prostheses’ clinical trials have shown that some of the participants cannot identify patterns or shapes. However, participants do benefit by their newly acquired ability to locate objects and detect motion in their daily activities. Yet there is no accepted method to quantify this acquired vision (Cohen, 2007). Due to the different pathologies of diseases that cause blindness, the outcome of a visual prosthesis is patient specific and thus, while some patients are able to identify patterns and can score on the extended visual acuity test, other patients can only locate objects or detect motion (Caspi et al., 2009; Humayun et al., 2012; Stingl et al., 2013). In order to gain a better understanding of the potential benefits of low resolution visual prostheses and to assess different image processing algorithms, visual prosthesis simulators are used. Generally, in a visual prosthesis simulator, also known as pixelized vision simulator, a real-time, low-resolution image of the view is presented on LCD goggles to a normally sighted user. The image of the scene is captured by a head mounted camera, digitalized by a computer, and a sub-sampled low resolution (‘‘pixelized’’) image is presented on a commercial eyewear video display (Fig. 1). A variety of tasks have been evaluated using pixelized vision simulator. Thompson et al. (2003) investigated the minimum requirements for face recognition and Fornos et al. (2005) used a visual prosthesis simulator to compare the scanning benefit and shape of individual pixels, square vs. Gaussian, in enabling reading. Hallum et al. (2005) explored the effect of pixelized vision on various eye movements, i.e. smooth pursuit, saccades, and fixation. Wang, Yang, and Dagnelie (2008a) investigated the effect of retinal location of the projected pixelized image on smooth pursuit initiation and stability. Dagnelie et al. (2007) and Wang, Yang, and Dagnelie (2008b) assessed virtual maze navigation and real mobility performance with simulated prosthetic vision. Parikh et al. (2013) compared various computer algorithms, including saliency-based cueing algorithms, using a visual prosthesis simulator. Clinical trials of visual prosthetics showed that some patients cannot perceive shapes. Published reports from 30 patients implanted with the Argus II prosthesis (Humayun et al., 2012) showed that only 23% can discriminate the orientation of a grating while 57% could discriminate motion and 96% of the patients were able to localize objects. Results from the 8 patients implanted with the Alpha IMS clinical trial (Stingl et al., 2013) showed that only 2/8 were able to score on the Landolt C test, 5/8 discriminate
Fig. 1. An image of the pixelized vision simulator which consists of a USB camera and miniature LCD monitors mounted on goggles.
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
gratings and identify motion, and 7/8 were able to performed a localization test. Although the testing methodologies were different in the two clinical trials, both reports hint that not all implanted patients see a pixelized distinct pattern. Therefore for a pixelized visual simulator to be a useful tool and accurately mimic retinal prostheses, they too must take into account the unpatterned perceived image. Herein, we used a pixelized visual simulator with both patterned and unpatterned images in order to evaluate if and under what conditions a measured visual acuity level is truly an indication that the visual prosthesis provides the patient with a patterned image. We compared the results of Landolt C visual acuity tests in two extreme conditions. In the first condition, visual acuity was measured using a classical pixelized image in which each pixel represented the brightness level in the receptive field of the pixel. This mapping reduces the resolution but preserves the stimulus pattern. In the second condition, we used a novel mapping which encoded only the total brightness in the field of view of the stimulus array. This mapping algorithm represents an unpatterned stimulus, hypothesized to occur in some patients implanted with retinal prostheses. In this mode there is no geometric resolution present in a static image and it would be impossible to derive any spatial information from such an image. Based solely on (non-existent) pattered information, patients with such vision should be unable to exhibit a measured visual acuity.
2. Methods 2.1. Pixelized vision simulator A Small, commercial USB WebCam (PK-836MJ, A4Tech, China) was attached to glasses that contained binocular Liquid Crystal Displays (AV230, Vuzix, USA) in order to provide an image with a resolution of 8 8 pixels. The WebCam was set to a resolution of 160 120 and images were sampled in real time by a Matlab application using the freeware vcapg2 function. The pixelized simulator was able to run in two modes, a Normal mode and a Brightness mode. In Normal Mode, a sub-sampled, 8 8, pixelized image with spatial information was created from the raw image by averaging the brightness in the corresponding field-ofview. Each pixel in the pixelized image was created by averaging the matching 15 15 pixels out of the 160 120 pixels in the raw image from the camera digitizing to a dynamic range of 5 levels. In this mode the resolution is set by the distance between neighboring pixels, which was 1.75°. This is equivalent to a stimulus of 2.0 log MAR (=log10 (60 1.75)). In Brightness Mode, the 8 8 pixelized image had the same brightness and size of the original image without any spatial patterns. It is worthwhile to mention that the perceived visual acuity is most likely better than the above resolution acuity (Heinrich & Bach, 2013). However, resolution acuity is usually given as a reference in order to evaluate the simulated implanted visual sensor. In order to create the unpatterned Brightness mode image, first a normal mode pixelized image was created and then the pixels were placed in the center in order to create a single structure without pattern. The unpatterned brightness mode image has the same overall brightness as the pixelized normal mode but without the pattern. The pixels were rearranged in the center of the image based on their brightness, the brightest pixels in the center. This creates a continuous, smooth percept. In this mode the entire simulated image is viewed as a ‘‘one-pixel’’ system and the resolution is set by the field of view. The total captured field-of-view of the 8 8 pixels is 14° which is 2.9 log MAR, which is 8 times larger than in the normal mode. Examples of each mode are shown in Figs. 2 and 3.
79
2.2. Visual acuity test Visual acuity was measured using the adaptive Freiburg Visual Acuity & Contrast Test (FrACT; Bach, 1996). The stimuli were Landolt-Cs in which the letter ‘‘C’’ is presented in one of 4 orientations and the subject needs to identify the side in which the gap appears. This is a freeware application that can be downloaded at: http:// www.michaelbach.de. The program was set to the 4 Alternative Force Choices (4AFC) task. In the FrACT program the visual acuity in each trial was set according to a maximum likelihood based on responses for all previous trials. Every sixth trial was an ‘‘easy’’ trial, in order to maintain the subject’s motivation. Maximum presentation time for each trial was 60 s, but concluded sooner if the subject reported his/her response earlier. 2.3. Subjects Six sighted volunteers, students aged 20–30 (3 female; average age 24), with normal or corrected to normal vision, participated in the experiment. The research was approved by the Ethics Committee of Bar Ilan University and was performed in accordance with the Declaration of Helsinki. Prior to the experiment each subject gave informed consent. Some of the subjects received course credit for participating in the study. All subject were naïve as to the aim of the experiment, but were subsequently informed about the goals and motivation of the study. 2.4. Procedure Visual acuities and reaction times of the subjects using the pixelized vision simulator in normal and brightness modes were measured using FrACT Landolt-C visual acuity test. Subjects sat in a darkened room in front of a 19 in. monitor at a 36 cm distance while wearing the Pixelized Vision Simulator. Initially the simulator was set to normal mode in which a low resolution, 8 8, subsampled image of the target was presented on the LCD of the glasses (snapshot in Fig. 2a) and subjects were accustomed to vision via the simulator by presenting gratings in various orientations. This was followed by the assessment of the visual acuity using the FrACT set to 30 trials at the normal mode of the simulator. The simulator was then switched to brightness mode in which a low resolution 8 8, pixelized image with the same brightness and size of the original image but without any spatial patterns was presented (see Fig. 2b) and the procedure for adapting and then assessing visual acuity was repeated. In both modes the subjects were permitted to move their head in order to scan the target by steering the line of sight of the camera. However, there were no explicit instructions regarding head movements and any such movements were due to the subject having ‘‘learned’’ that it can be of assistance. 2.5. Statistical analysis The validity of the FrACT Landolt-C visual acuity test was verified post hoc by analyzing the ensemble of the trials in the bracket of plus–minus 0.1 log MAR around the reported visual acuity of the adaptive algorithm. Using the cumulative binomial distribution we evaluated the probability that the results in this bracket were not driven by visual information, i.e. guessing. The binomial distribution was evaluated taking into account that on each trial the probability of reporting correctly by guessing is 0.25. Results were considered significant if the probability to score by guessing in the bracket of plus–minus 0.1 log MAR around the reported visual acuity of the adaptive algorithm was less than 0.01.
80
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
Fig. 2. Snapshots of the pixelized vision simulator in normal and brightness modes with a gap size that is equal to 1/2 the size of an electrode. The left column shows images of the stimulus as captured by the USB camera. The rectangle represents the field of view of the simulated prosthesis which is identical for all cases. The middle column shows the pixelized image in the normal mode. The Right column shows the pixelized image in brightness mode. Each row shows a different snapshot during scanning. It is noticeable that for the sub-pixel gap the optotype direction is not visible in all snapshots and hyper-acuity is achieved by scanning, which adds an additional dimension, that of time.
3. Results Visual acuities for the 6 subjects in the two modes with the Landolt-C test are shown in Fig. 4. The left column of Fig. 4 presents the results in normal mode while the result of the brightness mode is given in the right column. These acuities are summarized in Table 1. In the brightness mode one subject was not able to reach a level significantly greater than chance. All other results were significant with a probability greater than 0.99. In normal mode, visual acuity for all six subjects was at least 0.2 log MAR better than the geometrical resolution of the simulated pixelized vision (2.0 log MAR). The addition of a time dimension utilized by a head scanning strategy enabled them to achieve this acuity level. In brightness mode in which there is no patterned vision and performance is theoretically limited by the field of view
of the system, 4 out of the 6 subjects similarly adapted a head scanning strategy that enabled them to achieve an acuity level that is at least 0.5 log MAR better than the limit set by the field of view of the system (2.9 log MAR). Perhaps, with training subjects might perform better. Of the other two subjects, one achieved an acuity level equal to the geometrical limit and the other did not succeed in developing a scanning strategy and thus did not score above chance on any acuity level when the simulated vision did not contain a pattern. The response times of the subjects are given in Table 1. In brightness mode it took the subjects up to 3 times longer than in Normal mode to report their answer, although in Normal mode the response times were also longer than the time usually taken to respond to a visual acuity test. These times were due to the time required to scan the image in order to identify the location of the
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
81
Fig. 3. Snapshots of the pixelized vision simulator in normal and brightness modes with a gap size that is equal to half the field of view of the array. The left column shows images of the stimulus as captured by the USB camera. The rectangle represents the field of view of the simulated prosthesis which is identical for all cases. The middle column shows the pixelized image in the normal mode. The Right column shows the pixelized image in brightness mode. Each row shows a different snapshot during scanning. It is noticeable that in brightness mode it is impossible to detect the optotype from a single sample and visual functionality is achieved only by and because of the temporal scanning.
gap based on changes of the overall brightness in the field-of-view of the imager. 4. Discussion Herein we showed that in a manner similar to that observed with sighted individuals (Heinrich & Bach, 2013) simulated pixelized vision yields visual acuities better than the resolution acuity sets by the sensor. However, the time required for obtaining that acuity via a head scanning strategy is on the order of 10 s. Moreover, we showed that the measured perceptual acuity is better even in the absence of any resolution acuity as indicated by the Brightness Mode which required scanning time on the order of minutes. The results from the two modes offer insight into two phenomena that are observed in blind patients implanted with retinal
prostheses (Humayun et al., 2012). During Normal Mode we showed that while using a pixelized image simulator subjects can score acuity levels that are significantly better than would be expected based on the geometrical resolution of the system. In a previous study, a patient implanted with an epi-retinal prosthesis was able to score an acuity level that was significantly better than the limit set by the distance between neighboring electrodes (Humayun et al., 2012; Mohand-Said et al., 2011). Most likely, in that case the brain integrated information over time and used head scanning to achieve better than expected resolution, as seen in our subjects. This is similar to adding jitter to the image, a technique that is known to enhance visual performance when spatial resolution is limited (Watson et al., 2012). The benefit of head scanning in improving apparent visual acuity in pixeled vision system is long recognized. Already in 1992 Cha, Horch, and Normann (1992) anticipated the development of
82
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
Fig. 4. Visual acuity data of the 6 subjects using the pixelized image simulator in normal and brightness modes. Charts show the acuity level as a function of trial number. Left column panels show the results of runs in normal mode. Right column panels are the results of runs in normal mode. Filled circles represent a correct response, while a circle with an ‘‘X’’ represents a trial with an incorrect response. The estimated visual acuity, percent correct, is 62.5%, a mid-point between 100% and chance level (25%). Estimated visual acuity is indicated by the dashed line and the number to the right of the chart. In order to keep the subjects motivated, every sixth trial was an easy trial.
Table 1 Summary of the data. Table provides measured visual acuity and mean response time for each subject for both modes (Normal and Brightness). Subject ID
MSX-001 ASK-002 ASX-003 SHA-004 RBS-005 ADK-006
Gender
Male Female Male Female Male Female
Normal mode
Brightness mode
Visual acuity (log MAR)
Mean response time (sec)
Visual acuity (log MAR)
Mean response time (sec)
1.6 1.6 1.8 1.8 1.6 1.7
9.5 8.8 5.4 15.0 11.5 8.3
2.2 2.6 2.4 2.3 2.4 N.A.
21.0 39.0 16.8 41.8 28.1 25.5
visual prostheses and performed psychophysical experiments using a phosphene simulator on six subjects. They found that while neither eye movements nor smooth movements of the target affected visual acuity, ‘‘voluntary head movements did help our
subjects with the low pixel density masks. For example, our subjects showed a sudden improvement in performance right after they were encouraged to use head movements’’ (page 446). They postulate that this improvement may have been due to either
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
temporal integration while the object was scanned or the use of head movements to find the best viewing position. It should be noted that their experiments involved multiple sessions over a 3 month period. In contrast, we showed that subjects can adapt head scanning without extensive training, either to improve visual acuity in normal mode or to overcome the unpatterned image in brightness mode. In addition, all of their stimuli were pixelized, i.e. no equivalent of our brightness mode. Finally, we gave no instructions that would have encouraged head scanned; it seems that scanning is a natural process in the visual system that subjects employed on their own. More recently, Chen et al. (2006) built on Cha et al.’s findings and investigated what properties of head scanning optimize the scanning’s benefit. Using a simulation of prosthetic vision they had subjects perform ten sessions in which the phosphene lattice offered a theoretical equivalent of 2.0 log MAR. All subjects had a measured acuity greater than that thanks to head scanning. Furthermore, circular scanning seems to afford the greatest benefit, as does higher velocity scanning. As in our study, they offered no specific guidance. However, they too had no equivalent of our brightness mode as all stimuli were pixelized images that preserved the pattern of the image, which is not the case for many of the blind patients implanted with prostheses. Thus, our findings in ‘‘Normal Mode’’ are not totally unexpected. They extend previous findings regarding patterned vision in normally sighted individuals to a model of what retinal prosthetic patients with patterned vision might experience and introduce the importance of scanning to the literature dealing with the burgeoning field of visual prostheses. In Brightness Mode, in which the simulator delivered an image based only on intensity without any pattern information, subjects were able to score on the visual acuity test, but with a score that is still significantly worse (average of 2.2 log MAR) average than the spatial resolution set by the distance between pixels (2.0 log MAR), the equivalent of electrodes in prosthetic systems. Nonetheless, this is far better than when the system is looked upon as a ‘‘one-pixel’’ system, as it is in brightness mode. Note, that the brightness mode is not truly one pixel, because the width can change and it is not uniform. These results indicate that the subjects probably ‘‘saw’’ something and scored well above chance, which is better than would be expected from mere brightness information. This was most likely accomplished using a tedious and time consuming acquisition of information via a systematic scanning pattern. Support for this comes from the response time information. While a sighted subject in native vision responds in less than a second, the response using Normal Mode was on the order of 10 s (9.8 ± 3 s) and during Brightness Mode 2–3 times longer (28.7 ± 9.9 s). These findings regarding the brightness mode, i.e. unpatterned vision, are entirely novel and are very important in understanding the measured visual acuity and the observed visual functionality of individuals with retinal prostheses. Extended reaction time was found in a study of facial expression recognition using simulated prosthetic vision (van Rheede, Kennard, & Hicks, 2010). They employed different trade-offs between field of view and resolution and while there was a significant difference in the acuity that corresponded with the resolution, the reaction time for the facial recognition task was the same for all resolutions and field of view (approximately 4 s). We suggest that just as in our case, their extended reaction times are the result of strategies employed by the subjects of scanning in order to increase the usable information. In general, when clinical visual acuity (VA) is measured, time is not factored in. Most optometrists simply run through a standard test without giving a thought to how long the patient looks at each item. If a patient takes an inordinate amount of time to struggle
83
through a line of letters on an acuity chart, the clinician will usually not record that line as the patient’s visual acuity. On the other hand, with low vision patients, a clinician may opt to record either the acuity performed with ease or the one that the patient struggled with but was able to read with extra time. This second acuity number two reveals the potential of the low vision patient. In other words, normally sighted and visually impaired persons cannot be assessed in the same way. In testing anyone other than a ‘‘perfect’’ subject, such as a patient with artificial vision, time is indeed an important factor. Sometimes, for example, with a patient with albinism, an optometrist may subconsciously (or consciously) give them extra time, but that is then not taken into account in the final acuity measure. Clark and Clark (2013) showed that giving more time vastly improved the grating acuity for two cats. Yang et al. (2005) investigated the effect of limiting gazing time to 550 ms during an acuity test. They found that indeed for healthy subjects there was no difference in measured acuity between notime-restricted acuity and time-restricted acuity. However, for patients with Infantile Nystagmus Syndrome there was a significant improvement in reported acuity when they are given more time to examine each optotype. It is clear that individuals with compromised vision will compensate and develop time-consuming strategies to improve their acuity. In our case with a pixelized vision simulator, in both modes the acuity is severely limited, yet subjects developed strategies such that when given sufficient time would enable them to achieve a relatively high reportable acuity. This is despite the fact that they do not necessarily have that level of form vision. It is important that reports of visual acuities measured on blind with retinal prosthesis will report the response time or if time was limited. For example, in the Argus II clinical trials the reported visual acuities were measured with a time limit of five seconds (Humayun et al., 2012). Imposing a short time limit may increase the chances of measuring a ‘‘true’’ visual acuity. Or alternatively, it is important to report the actual reaction times together with the measured acuity. In conclusion, measurements of visual acuity are not necessarily an indication that the visual prosthesis provides an image with spatial patterns. Nevertheless, visual acuity levels at or better than the sub-pixel resolutions do seem to imply that the patient perceives an image with spatial patterns. Our study results show that there is not necessarily a one-to-one correspondence between visual acuity measurements and the exact spatial pattern produced by the retinal prosthesis device. We note that there is a difference between resolution acuity and recognition acuity. Vision with retinal prostheses – at least with those which can utilize scanning movements of either the eye or head – shows the known phenomenon that recognition acuity can be considerably better than expected from geometrical resolution acuity, and that this is similar to findings with Landolt C-ring based methods of VA assessment in normal vision. References Ahuja, A. K., Dorn, J. D., Caspi, A., McMahon, M. J., Dagnelie, G., Dacruz, L., et al. (2011). Argus II Study Group. Blind subjects implanted with the Argus II retinal prosthesis are able to improve performance in a spatial-motor task. British Journal of Ophthalmology, 95, 539–543. Bach, M. (1996). The ‘‘Freiburg Visual Acuity Test’’ – Automatic measurement of visual acuity. Optometry and Vision Science, 73, 49–53. Bach, M., Wilke, M., Wilhelm, B., Zrenner, E., & Wilke, R. (2010). Basic quantitative assessment of visual performance in patients with very low vision. Investigative Ophthalmology & Visual Science, 51, 1255–1260. Bailey, I. L., Jackson, A. J., Minto, H., Greer, R. B., & Chu, M. A. (2012). The Berkeley rudimentary vision test. Optometry and Vision Science, 89(9), 1257–1264. Caspi, A., Dorn, J. D., McClure, K. H., Humayun, M. S., Greenberg, R. J., & McMahon, M. J. (2009). Feasibility study of a retinal prosthesis: Spatial vision with a 16electrode implant. Archives of Ophthalmology, 127, 398–401.
84
A. Caspi, A.Z. Zivotofsky / Vision Research 108 (2015) 77–84
Cha, K., Horch, K., & Normann, R. A. (1992). Simulation of a phosphene-based visual field: Visual acuity in a pixelized vision system. Annals of Biomedical Engineering, 20(4), 439–449. Chen, S. C., Hallum, L. E., Suaning, G. J., & Lovell, N. H. (2006). Psychophysics of prosthetic vision: I. Visual scanning and visual acuity. Conference Proceedings on IEEE Engineering in Medicine and Biology Society, 1, 4400–4403. Clark, D. L., & Clark, R. A. (2013). The effects of time, luminance, and high contrast targets: Revisiting grating acuity in the domestic cat. Experimental Eye Research, 116, 75–78. Cohen, E. (2007). Safety and effectiveness considerations for clinical studies of visual prosthetic devices. Journal of Neural Engineering, 4, S124–S129. Dagnelie, G., Keane, P., Narla, V., Yang, L. C., Weiland, J., & Humayun, M. (2007). Real and virtual mobility performance in simulated prosthetic vision. Journal of Neural Engineering, 4(1), S92–S101. Dobson, V., Clifford-Donaldson, C. E., Miller, J. M., Garvey, K. A., & Harvey, E. M. (2009). A comparison of Lea symbol vs ETDRS letter distance visual acuity in a population of young children with a high prevalence of astigmatism. JAAPOS, 13, 253–257. Eiber, C. D., Lovell, N. H., & Suaning, G. J. (2013). Attaining higher resolution visual prosthetics: A review of the factors and limitations. Journal of Neural Engineering, 10(1). Falkenstein, I. A., Cochran, D. E., Azen, S. P., Dustin, L., Tammewar, A. M., Kozak, I., et al. (2008). Comparison of visual acuity in macular degeneration patients measured with snellen and early treatment diabetic retinopathy study charts. Ophthalmology, 115, 319–323. FDA Guidance for Industry and FDA Staff – Investigational Device Exemption (IDE) Guidance for Retinal Prostheses, 2013. Ferris, F. L., Kassoff, A., Bresnick, G. H., & Bailey, I. (1982). New visual acuity charts for clinical research. American Journal of Ophthalmology, 94, 91–96. Fornos, A. P., Sommerhalder, J., Rappaz, B., Safran, A. B., & Pelizzone, M. (2005). Simulation of artificial vision, III: Do the spatial or temporal characteristics of stimulus pixelization really matter? Investigative Ophthalmology & Visual Science, 46, 3906–3912. Hallum, L. E., Suaning, G. J., Taubman, D. S., & Lovell, N. H. (2005). Simulated prosthetic visual fixation, saccade, and smooth pursuit. Vision Research, 45, 775–788. Heinrich, S. P., & Bach, M. (2013). Resolution acuity versus recognition acuity with Landolt-style optotypes. Graefes Archive for Clinical and Experimental Ophthalmology, 251, 2235–2241. Hornig, R., Zehnder, T., Velikay-Parel, M., Laube, T., Feucht, M., & Richard, G. (2008). The IMI retinal implant system in artificial sight. Biological and Medical Physics, Biomedical Engineering, 111–128. Humayun, M. S., Dorn, J. D., da Cruz, L., Dagnelie, G., Sahel, J. A., Stanga, P. E., et al. (2012). Argus II Study Group interim results from the international trial of second sight’s visual prosthesis. Ophthalmology, 119, 779–788. Kobelt, G., Lundström, M., & Stenevi, U. (2002). Cost-effectiveness of cataract surgery. Method to assess cost-effectiveness using registry data. Journal of Cataract and Refractive Surgery, 28, 1742–1749. Kotecha, A., Zhong, J., Stewart, D., & da Cruz, L. (2014). The Argus II prosthesis facilitates reaching and grasping tasks: A case series. BMC Ophthalmology, 23(14), 71.
Lange, C., Feltgen, N., Junker, B., Schulze-Bonsel, K., & Bach, M. (2009). Resolving the clinical acuity categories ‘‘hand motion’’ and ‘‘counting fingers’’ using the Freiburg Visual Acuity Test (FrACT). Graefes Archive for Clinical and Experimental Ophthalmology, 247, 137–142. Mohand-Said, S., Caspi, A., Merlini, F., et al. (2011). Comparison of Etdrs, Landolt C, and grating visual acuity tests between sighted volunteers using a pixelized image simulator and blind subjects implanted with the ArgusTm Ii retinal prosthesis. Investigative Ophthalmology & Visual Science, 52. E-Abstract 4931. Nau, A., Bach, M., & Fisher, C. (2013). Clinical tests of ultra-low vision used to evaluate rudimentary visual perceptions enabled by the brainport vision device. Translational Vision Science & Technology, 2(3), 1. Nau, A. C., Pintar, C., Fisher, C., Jeong, J. H., & Jeong, K. (2014). A standardized obstacle course for assessment of visual function in ultra low vision and artificial vision. Journal of Visualized Experiments, 84. Papsin, B. C., & Gordon, K. A. (2007). Cochlear implants for children with severe-toprofound hearing loss. New England Journal of Medicine, 357, 2380–2387. Parikh, N., Itti, L., Humayun, M., & Weiland, J. (2013). Performance of visually guided tasks using simulated prosthetic vision and saliency-based cues. Journal of Neural Engineering, 10(2). Rosenfeld, P. J., Brown, D. M., Heier, J. S., Boyer, D. S., Kaiser, P. K., Chung, C. Y., et al. (2006). MARINA Study Group Ranibizumab for neovascular age-related macular degeneration. New England Journal of Medicine, 355, 1419–1431. Schulze-Bonsel, K., Feltgen, N., Burau, H., Hansen, L., & Bach, M. (2006). Visual acuities ‘‘hand motion’’ and ‘‘counting fingers’’ can be quantified with the Freiburg Visual Acuity Test. Investigative Ophthalmology & Visual Science, 47, 1236–1240. Stingl, K., Bartz-Schmidt, K. U., Besch, D., Braun, A., Bruckmann, A., Gekeler, F., et al. (2013). Artificial vision with wirelessly powered subretinal electronic implant alpha-IMS. Proceedings of the Royal Society B: Biological Sciences, 280(1757). Thompson, R. W., Barnett, G. D., Humayun, M. S., & Dagnelie, G. (2003). Facial recognition using simulated prosthetic pixelized vision. Investigative Ophthalmology & Visual Science, 44, 5035–5042. van Rheede, J. J., Kennard, C., & Hicks, S. L. (2010). Simulating prosthetic vision: Optimizing the information content of a limited visual display. Journal of Vision, 10(14). Wang, L., Yang, L. C., & Dagnelie, G. (2008a). Initiation and stability of pursuit eye movements in simulated retinal prosthesis at different implant locations. Investigative Ophthalmology & Visual Science, 49, 3933–3939. Wang, L., Yang, L. C., & Dagnelie, G. (2008b). Virtual way finding using simulated prosthetic vision in Gaze-locked viewing. Optometry and Vision Science, 85, 1057–1063. Watson, L. M., Strang, N. C., Scobie, F., Love, G. D., Seidel, D., & Manahilov, V. (2012). Image jitter enhances visual performance when spatial resolution is impaired. Investigative Ophthalmology & Visual Science, 53, 6004–6010. Yang, D., Hertle, R. W., Hill, V. M., & Stevens, D. J. (2005). Gaze-dependent and timerestricted visual acuity measures in patients with Infantile Nystagmus Syndrome (INS). American Journal of Ophthalmology, 139, 716–718. Zrenner, E., Bartz-Schmidt, K. U., Benav, H., Besch, D., Bruckmann, A., Gabel, V. P., et al. (2011). Subretinal electronic chips allow blind patients to read letters and combine them to words. Proceedings of the Royal Society B: Biological Sciences, 278(1711), 1489–1497.
