Curr Diab Rep (2015) 15:14 DOI 10.1007/s11892-015-0577-6
MICROVASCULAR COMPLICATIONS—RETINOPATHY (JK SUN, SECTION EDITOR)
Automated Retinal Image Analysis for Diabetic Retinopathy in Telemedicine Dawn A. Sim & Pearse A. Keane & Adnan Tufail & Catherine A. Egan & Lloyd Paul Aiello & Paolo S. Silva
# Springer Science+Business Media New York 2015
Abstract There will be an estimated 552 million persons with diabetes globally by the year 2030. Over half of these individuals will develop diabetic retinopathy, representing a nearly insurmountable burden for providing diabetes eye care. Telemedicine programmes have the capability to distribute quality eye care to virtually any location and address the lack of access to ophthalmic services. In most programmes, there is currently a heavy reliance on specially trained retinal image graders, a resource in short supply worldwide. These factors necessitate an image grading automation process to increase the speed of retinal image evaluation while maintaining accuracy and cost effectiveness. Several automatic retinal image analysis systems designed for use in telemedicine have recently become commercially available. Such systems have the potential to substantially improve the manner by which diabetes eye care is delivered by providing automated real-time evaluation to expedite diagnosis and referral if required. Furthermore, integration with electronic medical records may This article is part of the Topical Collection on Microvascular Complications—Retinopathy D. A. Sim : L. P. Aiello : P. S. Silva (*) Department of Ophthalmology, Harvard Medical School and Beetham Eye Institute, Joslin Diabetes Center, One Joslin Place, Boston, MA 02215, USA e-mail: [email protected] D. A. Sim : P. A. Keane : A. Tufail : C. A. Egan NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust, London, UK D. A. Sim : P. A. Keane : A. Tufail Institute of Ophthalmology, University College London, London, UK P. S. Silva Teleophthalmology and Image Reading Center, Philippine Eye Research Institute, National Institutes of Health, University of the Philippines, Manila, Philippines
allow a more accurate prognostication for individual patients and may provide predictive modelling of medical risk factors based on broad population data. Keywords Automated retinal image analysis . Telemedicine . Diabetes mellitus . Diabetic retinopathy Abbreviations ARIA Automated retinal image analysis JVN Joslin Vision Network CADe Computer-aided detection CADx Computer-aided diagnosis ETDRS Early treatment diabetic retinopathy study ARIS Automated Retinal Imaging System PCA Principal component analysis DRS Diabetic retinopathy study
Introduction Eduard Jaeger [1] in 1856 first described the controversial observations of “yellowish spots and extravasations” in the macula of a diabetic patient. It would take over 113 years before the Airlie House Symposium on the Treatment of Diabetic Retinopathy established basis for the current photographic method of quantifying the presence and severity of diabetic retinopathy. The characteristic lesions of diabetic retinopathy may develop in anyone with diabetes mellitus, and these lesions are presently estimated to affect nearly half of those diagnosed with diabetes at any given time [2–5]. The advances in the medical management of diabetes that begun with discovery of insulin by Frederick Banting, Charles Best and colleagues in 1921 [6] have substantially increased patient survival and life expectancy. Patients with diabetes are now living longer; however, in doing so, they are at increased risk
14
Page 2 of 9
for developing diabetes-related microvascular complications, the most common of which is diabetic retinopathy [4, 5]. The current treatments available for diabetic retinopathy are highly effective in preventing visual loss [7–10]. Early detection and accurate evaluation of diabetic retinopathy severity, coordinated medical care and prompt appropriate treatment represent an effective paradigm of diabetic eye care. However, current estimates suggest that access to such care in developed countries ranges between 60 and 90%, with significantly lower rates in developing countries [11]. Without access to eye care, individuals do not benefit from these remarkably effective therapeutic advances achieved over the past century. Telemedicine programmes addressing diabetic retinopathy have the potential to distribute quality eye care to virtually any location and address the lack of access to eye care. In England and Wales, a systematic population-based diabetic retinopathy screening programme together with documented improvements in diabetes medical care may have, for the first time in at least five decades, resulted in diabetic retinopathy no longer being the leading cause of blindness among working age adults [12]. However, with a projected 552 million individuals with diabetes globally by the year 2030, current telemedicine programmes will require technical enhancements addressing image acquisition, automated image analysis algorithms and predictive biomarkers to successfully manage this epidemic. This review evaluates the current state-of-the-art in retinal image analysis for diabetic retinopathy telemedicine programmes.
Potential Impact of Automated Retinal Image Analysis on Diabetic Retinopathy Telemedicine Programmes With the aid of digital retinal colour photography, telemedicine has allowed for timely and accurate detection of diabetic retinopathy, especially for populations where eye care delivery was not previously feasible. Economic analysis of teleophthalmology programmes for diabetic retinopathy based on the Joslin Vision Network (JVN) used by three US federal health care agencies found these telemedicine efforts to be a less costly and a more effective strategy than conventional clinic-based ophthalmoscopy for identifying and accurately determining diabetic retinopathy severity and preventing cases of severe vision loss [13]. Moreover, the implementation of the JVN programme in primary care settings over a 5-year period has resulted in a 50 % increase in diabetic retinopathy surveillance rate and a proportional 50 % increase in the rate of laser treatment [14]. Telemedicine programmes for diabetic retinopathy include the following key clinical components: (1) a remote, reliable, cost-effective image acquisition system capable of reproducibly acquiring high quality retinal images; (2) an image reading centre for retinal image analysis to assess diabetic retinopathy severity; and (3) a clinical coordinating centre that
Curr Diab Rep (2015) 15:14
communicates the findings to primary care providers and/or patients and facilitates clinic appointments or therapy as required. In addition, telemedicine programmes require expensive technical, information technology and administrative support. Nonetheless, one of the main barriers towards a wider implementation of teleophthalmology has remained the requirement and costs of trained personnel for image reading and grading of a large volume of retinal images. Since its first deployment 10 years ago, the JVN has provided diabetes care to over 100,000 patients, and the current database servers contain over 2 million retina images [15]. The Indian Health Service–JVN Teleophthalmology Program generated retinal images from 1624 patients in 2003 [14]. This has now grown to 14,804 patients per year in 2013 with an average grading time of 10 min per patient. A total of at least 2500 man-hours per year will be required to grade diabetic retinopathy severity from retinal images within this programme alone. In the UK (2010 to 2011), 1.9 million people were evaluated under the National Health Service Diabetic Eye Screening Program [16]. With a surveillance rate of over 90 %, the UK programme generates retinal images from at least 1.7 million patients per a year that have to be graded manually with an estimated equivalent workload of more than 300,000 man-hours per year. Reading retinal images is a highly skilled process, which requires training, continual quality control, maintenance of a specialised skill set, and heavy reliance on the experience and knowledge of the individual reader. Since trained retinal image readers are expensive and in limited supply worldwide, it has become a necessity to seek automation processes in ocular telemedicine in order to increase throughput while maintaining cost effectiveness and accuracy. Importantly, automation would also benefit the patient by permitting more rapid diagnosis and the potential for more time devoted to patient education and possibly real-time communication with the primary care physician. For the foreseeable future, human readers will be required for the purpose of quality control, adjudication, and interpretation of atypical retinal images. However, rather than fully replace the human reader, ARIA could improve workflow, diminish observer fatigue and reduce bias while performing important initial triage of low risk images. Ideally, the ARIA system should link a person’s electronic medical record with their retinal images before performing automated grading and diagnosis at the time of imaging. This will allow relevant individualised medical information to be considered in the stratification of an individual’s risk profile and facilitate counselling and treatment or follow-up planning during the imaging visit. Various ARIA strategies for assessing diabetic retinopathy severity have been extensively evaluated [17–20]. In this review, we examine the potential impact of automated image analysis on ocular telemedicine for diabetic retinopathy, summarise the different ARIA approaches, and evaluate the suitability of the current ARIA programmes for this indication.
Curr Diab Rep (2015) 15:14
Approaches to Automated Retinal Image Analysis Since the first stereoscopic fundus photograph by Jackson and Weber in 1886, which required a camera fixed onto the patient’s head and a 2.5-min exposure, the documentation of anatomy and pathological changes in the human retina has been a subject of interest to ophthalmologists, artists, and engineers alike. However, it was the introduction of highresolution digital retinal imaging systems in the 1990s combined with exceptional growth in computing power that permitted the development of computer algorithms capable of computer-aided detection (CADe), and computer-aided diagnosis (CADx). CADe is the identification of pathologic lesions. CADx provides a classification that incorporates additional lesion or clinical information to stratify risk or estimate the probability of disease. These recent imaging and computing advances allowed ARIA, a topic of interest for more than 40 years, to finally realistically address clinical applications over the past decade. A key component to the telemedicine approach is the clinical validation of retinal image analyses against the current gold-standard established by the Early Treatment Diabetic Retinopathy Study (ETDRS) consisting of 30°, stereoscopic, seven-standard field, colour 35-mm slides [21]. The American Telemedicine Associated has published position statements in order to provide standards and guidelines specifically for diabetic retinopathy [22, 23] and recommends that automated algorithms be compared to the accuracy and precision provided by the ETDRS imaging and evaluation standards. Over that past decade, there has been a host of algorithms developed for automated detection of diabetic retinopathy, the details of which has been reviewed elsewhere [17, 18, 24]. Broadly, the approach to ARIA can be categorised into two components: (1) image quality assessment and (2) image analysis. Image Quality Assessment Only recently have digital cameras approached the resolution attained by their 35-mm film counterparts. Independent comparative validation studies [25, 26] and analysis from multicentre clinical trials [27, 28] have reported good to excellent agreement in comparing diabetic retinopathy severity on film and digital images, indicating that the use of digital images at their current resolution do not systematically alter detection of diabetic retinopathy severity. In general, the agreement between film and digital images is less precise at two particular points on the severity scale, first in determining the presence of mild nonproliferative retinopathy (driven primarily by microaneurysms) and at moderate nonproliferative diabetic retinopathy (driven primarily by intraretinal microvascular abnormalities) [27]. Although modern electronic sensors have low image noise, they can be susceptible to pattern artefacts due to the grid arrangement of digital sensors. The
Page 3 of 9 14
addition of Bayer filters is helpful in reducing noise in the image. Currently, retinal digital photography has progressed to a stage where colour retinal photographs can be obtained using low levels of illumination through an undilated pupil. [29, 30]. However, human factors such as movement and positioning and ocular factors like cataract and reflections from retinal tissues can produce artefacts. Without pupillary dilation, artefacts are observed in 3–30 % of retinal images to an extent where they impede human grading [29, 31]. Thus, the importance of good image quality prior to automated image analysis has been recognised, and much ancillary research has been conducted in the field of image pre-processing. In general, this consists at least of comparing the histogram of an image obtained to that of an ideal histogram describing the brightness, contrast and signal/noise ratio, and/or determination of image clarity by assessing vessels surrounding the macula [32]. Image acquisition can also be automated, but the rate of ungradable images needs to be closely evaluated. The Automated Retinal Imaging System (ARISTM, Visual Pathways, Inc., Prescott, AZ, USA) [33] and Centervue DRS (Centervue SpA, Padova, Italy) [34] are imaging systems that may be programmed to acquire sequential colour digital images of defined fundus fields using minimally trained technicians. However, these systems have ungradable rates for individual images as high as 30 %. The automated image acquisition in these systems would be helpful in obtaining consistent fields using minimally trained technician; however, this would need to be combined with image quality analysis to determine the need for re-imaging a field. Until much improved automated acquisition systems are available, image pre-processing will likely need to be incorporated into automated image acquisition retinal cameras, allowing real-time assessment of image quality, providing real-time feedback to the technician and allowing a patient to be re-imaged at the same visit if the retinal image does not meet a given standard. Real-time image quality assessment systems are not yet available commercially. One such a system is being developed by VisionQuest Biomedical, LLC, Albuquerque, NM, USA, and has been validated on 2000 colour retinal images showing 100 % sensitivity and 96 % specificity in identifying “rejected” images [35]. Image Analysis An important first step of ARIA for diabetic retinopathy begins with identification and localisation of normal anatomy, better known as “segmentation”, of structures such as the optic nerve head, fovea and large retinal vessels. The main purposes of identifying these structures are to exclude them in the analysis for abnormal lesions and also to determine their locations for image registration (e.g. alignment with a reference image or a previously acquired image) and for use as a reference for
14
Curr Diab Rep (2015) 15:14
Page 4 of 9
measuring distances within the image. Localisation of the optic nerve head is a well-established approach. Some examples of techniques used in this regard include the identification of high pixel intensity within a retinal image (the optic disc is usually the brightest “spot” in a retinal image), principal component analysis (PCA; a method used for face recognition software that differentiates the optic disc from other bright spots such as exudates or reflections in the retina), the Hough transform (which utilises shape orientations to plot locations) and geometrical parametric modelling (a technique that exploits patterns from which retinal vessels emerge from the optic disc) [19, 24]. Automated detection of abnormal diabetic retinopathy lesions was initially performed on fluorescein angiograms [36]. Although fluorescein provides a background with high contrast between vascular and nonvascular structures and allows easier identification of vascular pathology, the requirement of intravenous infusion precluded its wide-scale application. The development of ARIA for diabetic retinopathy has therefore largely focussed on the use of digital colour images of the retina. The main challenge encountered with processing of colour images is the presence of numerous “distractors” within the retinal image (retinal capillaries, underlying choroidal vessels, and reflection artefacts), all of which may be confused with diabetic retinopathy lesions. As a result, much research has been focussed on the selective identification of diabetic retinopathy features, including microaneurysm s, heamorrhages, hard or soft exudates, cotton-wool spots and venous beading. These clinical features have been described in great detail in the landmark clinical trials, Diabetic Retinopathy Study (DRS) and ETDRS. [21, 37]. Earlier work in ARIA utilised basic image processing techniques such as thresholding, edge detection, filters and morphological processing [24]. These techniques focussed on identifying individual diabetic retinopathy features mimicking what is typically performed with manual human grading of individual lesions. Techniques on the extraction of microaneurysms from colour fundus images by the process of thresholding after excluding normal anatomical structures have been extensively reported with varying levels of success [18]. Recent developments in ARIA have moved away from individual detectors to approaches that include the following: an ensemble-based approach (integrates components of microaneurysm detectors) [38], a multiple-lesion approach (fuses multiple classifiers) [39], intelligent systems (which require training algorithms with data derived from human operators) [40] and content-based image retrieval techniques (analyses the contents of large digital image databases and compares it with an archive of labelled images) [41]. The details of algorithms and mathematical modelling methods used and their comparative efficiencies have been published in several reviews [17–19, 24]. These emerging trends of ARIA may prove most relevant to telemedicine given their
potential to go beyond traditional retinal image analysis, possibly providing more detailed assessment of retinopathy severity and greater accuracy of progression risk. Addition of electronic medical record integration may not only allow more accurate prognostication for individual patients but also provide predictive modelling of risk factors based on broad population data.
Overview of ARIA Systems Currently Deployed in Telemedicine and Screening Programmes Current commercially available systems that have been used in telemedicine or screening programmes include the iGradingM (Medalytix Group Ltd), the TRIADTM Network (Hubble Telemedical, Inc.), Iowa Detection Program (IDx, LLC), RetmarkerDR (Retmarker Ltd, Coimbra, Portugal) and Retinalyze System (Retinalyze A/S, Hørsholm, Denmark) (Table 1). A comparison of published sensitivity and specificity of each programme as well as a brief description of the associated published studies are presented in Table 2. Direct head-to-head comparisons between systems have proven difficult, mainly because of different photographic protocols, algorithms and patient populations used for validation. A common thread amongst these automated systems is to identify referable retinopathy: diabetic retinopathy, which requires the attention of an ophthalmologist. To date, we have not progressed to where human intervention can be fully removed from such programmes. All of the systems described below are semi-automated at some point in the workflow pathway and require assistance of a human reader/grader. iGradingM iGradingM is a product of Medalytix Group Ltd, which received its class 1 Conformité Européenne (CE) mark in 2013 and performs “disease/no disease” grading for diabetic retinopathy [42, 43]. It was developed at the University of Aberdeen in Scotland and uses previously published algorithms to assess both image quality and disease and has been described in detail elsewhere [32, 44]. A previously trained automated classifier on a set of 35 images containing 198 individually annotated microaneurysm or dot haemorrhages was used in its development. It was first deployed on a large-scale population in the Scottish Diabetic Retinopathy Screening Program in 2010, after being validated using several large screening populations in Scotland [39, 43, 45•, 46, 47], the largest being 78,601 single-field 45° colour fundus images from 33,535 consecutive patients [45•]. In this retrospective study, 6.6 % of the cohort had referable retinopathy, and iGradingM attained a sensitivity of 97.8 % for referable retinopathy. Although the specificity for referable retinopathy was not reported in the paper, it has since been calculated at 41.1 % [20]. iGradingM
Curr Diab Rep (2015) 15:14 Table 1
Page 5 of 9 14
Overview of current ARIA systems developed for diabetic retinopathy in telemedicine
ARIA system
Company
Developed at
Grading details
Current deployment
iGradingM
Medalytix LLC; Digital Healthcare
University of Aberdeen, Scotland, UK
Diabetic retinopathy absent/ diabetic retinopathy present
The TRIADTM Network
Hubble Telemedicine Inc.
University of Tennessee Health Science Center and the Oak Ridge National Laboratory, USA
IDx-DR
IDx LLC
University of Iowa, USA
RetmarkerDR
Retmarker Ltd.
Coimbra University, Portugal
CBIR technology not approved for use; diagnosis diabetic retinopathy severity with a supervising retinal specialist Diabetic retinopathy index; referable retinopathy (more than mild nonproliferative diabetic retinopathy) Diabetic retinopathy absent/ diabetic retinopathy present; microaneurysm turnover
CE mark 2013 as a class 1 medical device; Scottish Diabetic Retinopathy Screening Program Primary care services, mid-south region of the USA
RetinaLyze A/S
RetinaLyze A/S
Denmark
Microaneurysm and haemorrhage detection
has been further validated on 8271 screening episodes from a South London population in the UK [48]. Here, there was a higher percentage (7.1 %) of referable disease, and a sensitivity of 97.4–99.1 % and specificity of 98.3–99.3 % were attained depending on the configuration used. The TRIAD™ Network The TRIAD™ Network (Telemedical Retinal Image Analysis and Diagnosis) was launched commercially in 2012 by its developers at Hubble Telemedical Inc., and was developed at the University of Tennessee Health Science Center and the Oak Ridge National Laboratory [49]. It consists of a web-based technical network infrastructure that has been functional as a telemedicine network for diabetic retinopathy screening in the mid-south region of the USA since 2009 [50, 51]. ARIA was introduced to the TRIAD™ Network over a period of time, beginning with image quality analysis at the point of image capture [52] and most recently with the use of their patented content-based image retrieval techniques for automated diagnosis [53•]. The technique Table 2
CE mark 2013 as a class IIa medical device
CE mark 2011 as a Class IIa medical device; diabetic retinopathy screening programmes in Portugal CE mark 2011 as a class IIa medical device; software re-launched commercially 2013
of content-based image retrieval involves the comparison of images to large database collections using pictorial content. These image features include various description models, perceptual organisation, spatial relationships and may including clinical metadata [53•]. The authors themselves have reviewed challenges that they have faced with validation of their ARIA software, due mostly to the lack of large, varied and publically available datasets [50]. Briefly, they note that TRIADTM Network infrastructure has created a useful internal validation mechanism by which new algorithms can be developed and tested on datasets from different time points [53•, 54, 55]. Although external validation has been performed for anomaly detection using publically available datasets, these only contain a small number of images (
MICROVASCULAR COMPLICATIONS—RETINOPATHY (JK SUN, SECTION EDITOR)
Automated Retinal Image Analysis for Diabetic Retinopathy in Telemedicine Dawn A. Sim & Pearse A. Keane & Adnan Tufail & Catherine A. Egan & Lloyd Paul Aiello & Paolo S. Silva
# Springer Science+Business Media New York 2015
Abstract There will be an estimated 552 million persons with diabetes globally by the year 2030. Over half of these individuals will develop diabetic retinopathy, representing a nearly insurmountable burden for providing diabetes eye care. Telemedicine programmes have the capability to distribute quality eye care to virtually any location and address the lack of access to ophthalmic services. In most programmes, there is currently a heavy reliance on specially trained retinal image graders, a resource in short supply worldwide. These factors necessitate an image grading automation process to increase the speed of retinal image evaluation while maintaining accuracy and cost effectiveness. Several automatic retinal image analysis systems designed for use in telemedicine have recently become commercially available. Such systems have the potential to substantially improve the manner by which diabetes eye care is delivered by providing automated real-time evaluation to expedite diagnosis and referral if required. Furthermore, integration with electronic medical records may This article is part of the Topical Collection on Microvascular Complications—Retinopathy D. A. Sim : L. P. Aiello : P. S. Silva (*) Department of Ophthalmology, Harvard Medical School and Beetham Eye Institute, Joslin Diabetes Center, One Joslin Place, Boston, MA 02215, USA e-mail: [email protected] D. A. Sim : P. A. Keane : A. Tufail : C. A. Egan NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust, London, UK D. A. Sim : P. A. Keane : A. Tufail Institute of Ophthalmology, University College London, London, UK P. S. Silva Teleophthalmology and Image Reading Center, Philippine Eye Research Institute, National Institutes of Health, University of the Philippines, Manila, Philippines
allow a more accurate prognostication for individual patients and may provide predictive modelling of medical risk factors based on broad population data. Keywords Automated retinal image analysis . Telemedicine . Diabetes mellitus . Diabetic retinopathy Abbreviations ARIA Automated retinal image analysis JVN Joslin Vision Network CADe Computer-aided detection CADx Computer-aided diagnosis ETDRS Early treatment diabetic retinopathy study ARIS Automated Retinal Imaging System PCA Principal component analysis DRS Diabetic retinopathy study
Introduction Eduard Jaeger [1] in 1856 first described the controversial observations of “yellowish spots and extravasations” in the macula of a diabetic patient. It would take over 113 years before the Airlie House Symposium on the Treatment of Diabetic Retinopathy established basis for the current photographic method of quantifying the presence and severity of diabetic retinopathy. The characteristic lesions of diabetic retinopathy may develop in anyone with diabetes mellitus, and these lesions are presently estimated to affect nearly half of those diagnosed with diabetes at any given time [2–5]. The advances in the medical management of diabetes that begun with discovery of insulin by Frederick Banting, Charles Best and colleagues in 1921 [6] have substantially increased patient survival and life expectancy. Patients with diabetes are now living longer; however, in doing so, they are at increased risk
14
Page 2 of 9
for developing diabetes-related microvascular complications, the most common of which is diabetic retinopathy [4, 5]. The current treatments available for diabetic retinopathy are highly effective in preventing visual loss [7–10]. Early detection and accurate evaluation of diabetic retinopathy severity, coordinated medical care and prompt appropriate treatment represent an effective paradigm of diabetic eye care. However, current estimates suggest that access to such care in developed countries ranges between 60 and 90%, with significantly lower rates in developing countries [11]. Without access to eye care, individuals do not benefit from these remarkably effective therapeutic advances achieved over the past century. Telemedicine programmes addressing diabetic retinopathy have the potential to distribute quality eye care to virtually any location and address the lack of access to eye care. In England and Wales, a systematic population-based diabetic retinopathy screening programme together with documented improvements in diabetes medical care may have, for the first time in at least five decades, resulted in diabetic retinopathy no longer being the leading cause of blindness among working age adults [12]. However, with a projected 552 million individuals with diabetes globally by the year 2030, current telemedicine programmes will require technical enhancements addressing image acquisition, automated image analysis algorithms and predictive biomarkers to successfully manage this epidemic. This review evaluates the current state-of-the-art in retinal image analysis for diabetic retinopathy telemedicine programmes.
Potential Impact of Automated Retinal Image Analysis on Diabetic Retinopathy Telemedicine Programmes With the aid of digital retinal colour photography, telemedicine has allowed for timely and accurate detection of diabetic retinopathy, especially for populations where eye care delivery was not previously feasible. Economic analysis of teleophthalmology programmes for diabetic retinopathy based on the Joslin Vision Network (JVN) used by three US federal health care agencies found these telemedicine efforts to be a less costly and a more effective strategy than conventional clinic-based ophthalmoscopy for identifying and accurately determining diabetic retinopathy severity and preventing cases of severe vision loss [13]. Moreover, the implementation of the JVN programme in primary care settings over a 5-year period has resulted in a 50 % increase in diabetic retinopathy surveillance rate and a proportional 50 % increase in the rate of laser treatment [14]. Telemedicine programmes for diabetic retinopathy include the following key clinical components: (1) a remote, reliable, cost-effective image acquisition system capable of reproducibly acquiring high quality retinal images; (2) an image reading centre for retinal image analysis to assess diabetic retinopathy severity; and (3) a clinical coordinating centre that
Curr Diab Rep (2015) 15:14
communicates the findings to primary care providers and/or patients and facilitates clinic appointments or therapy as required. In addition, telemedicine programmes require expensive technical, information technology and administrative support. Nonetheless, one of the main barriers towards a wider implementation of teleophthalmology has remained the requirement and costs of trained personnel for image reading and grading of a large volume of retinal images. Since its first deployment 10 years ago, the JVN has provided diabetes care to over 100,000 patients, and the current database servers contain over 2 million retina images [15]. The Indian Health Service–JVN Teleophthalmology Program generated retinal images from 1624 patients in 2003 [14]. This has now grown to 14,804 patients per year in 2013 with an average grading time of 10 min per patient. A total of at least 2500 man-hours per year will be required to grade diabetic retinopathy severity from retinal images within this programme alone. In the UK (2010 to 2011), 1.9 million people were evaluated under the National Health Service Diabetic Eye Screening Program [16]. With a surveillance rate of over 90 %, the UK programme generates retinal images from at least 1.7 million patients per a year that have to be graded manually with an estimated equivalent workload of more than 300,000 man-hours per year. Reading retinal images is a highly skilled process, which requires training, continual quality control, maintenance of a specialised skill set, and heavy reliance on the experience and knowledge of the individual reader. Since trained retinal image readers are expensive and in limited supply worldwide, it has become a necessity to seek automation processes in ocular telemedicine in order to increase throughput while maintaining cost effectiveness and accuracy. Importantly, automation would also benefit the patient by permitting more rapid diagnosis and the potential for more time devoted to patient education and possibly real-time communication with the primary care physician. For the foreseeable future, human readers will be required for the purpose of quality control, adjudication, and interpretation of atypical retinal images. However, rather than fully replace the human reader, ARIA could improve workflow, diminish observer fatigue and reduce bias while performing important initial triage of low risk images. Ideally, the ARIA system should link a person’s electronic medical record with their retinal images before performing automated grading and diagnosis at the time of imaging. This will allow relevant individualised medical information to be considered in the stratification of an individual’s risk profile and facilitate counselling and treatment or follow-up planning during the imaging visit. Various ARIA strategies for assessing diabetic retinopathy severity have been extensively evaluated [17–20]. In this review, we examine the potential impact of automated image analysis on ocular telemedicine for diabetic retinopathy, summarise the different ARIA approaches, and evaluate the suitability of the current ARIA programmes for this indication.
Curr Diab Rep (2015) 15:14
Approaches to Automated Retinal Image Analysis Since the first stereoscopic fundus photograph by Jackson and Weber in 1886, which required a camera fixed onto the patient’s head and a 2.5-min exposure, the documentation of anatomy and pathological changes in the human retina has been a subject of interest to ophthalmologists, artists, and engineers alike. However, it was the introduction of highresolution digital retinal imaging systems in the 1990s combined with exceptional growth in computing power that permitted the development of computer algorithms capable of computer-aided detection (CADe), and computer-aided diagnosis (CADx). CADe is the identification of pathologic lesions. CADx provides a classification that incorporates additional lesion or clinical information to stratify risk or estimate the probability of disease. These recent imaging and computing advances allowed ARIA, a topic of interest for more than 40 years, to finally realistically address clinical applications over the past decade. A key component to the telemedicine approach is the clinical validation of retinal image analyses against the current gold-standard established by the Early Treatment Diabetic Retinopathy Study (ETDRS) consisting of 30°, stereoscopic, seven-standard field, colour 35-mm slides [21]. The American Telemedicine Associated has published position statements in order to provide standards and guidelines specifically for diabetic retinopathy [22, 23] and recommends that automated algorithms be compared to the accuracy and precision provided by the ETDRS imaging and evaluation standards. Over that past decade, there has been a host of algorithms developed for automated detection of diabetic retinopathy, the details of which has been reviewed elsewhere [17, 18, 24]. Broadly, the approach to ARIA can be categorised into two components: (1) image quality assessment and (2) image analysis. Image Quality Assessment Only recently have digital cameras approached the resolution attained by their 35-mm film counterparts. Independent comparative validation studies [25, 26] and analysis from multicentre clinical trials [27, 28] have reported good to excellent agreement in comparing diabetic retinopathy severity on film and digital images, indicating that the use of digital images at their current resolution do not systematically alter detection of diabetic retinopathy severity. In general, the agreement between film and digital images is less precise at two particular points on the severity scale, first in determining the presence of mild nonproliferative retinopathy (driven primarily by microaneurysms) and at moderate nonproliferative diabetic retinopathy (driven primarily by intraretinal microvascular abnormalities) [27]. Although modern electronic sensors have low image noise, they can be susceptible to pattern artefacts due to the grid arrangement of digital sensors. The
Page 3 of 9 14
addition of Bayer filters is helpful in reducing noise in the image. Currently, retinal digital photography has progressed to a stage where colour retinal photographs can be obtained using low levels of illumination through an undilated pupil. [29, 30]. However, human factors such as movement and positioning and ocular factors like cataract and reflections from retinal tissues can produce artefacts. Without pupillary dilation, artefacts are observed in 3–30 % of retinal images to an extent where they impede human grading [29, 31]. Thus, the importance of good image quality prior to automated image analysis has been recognised, and much ancillary research has been conducted in the field of image pre-processing. In general, this consists at least of comparing the histogram of an image obtained to that of an ideal histogram describing the brightness, contrast and signal/noise ratio, and/or determination of image clarity by assessing vessels surrounding the macula [32]. Image acquisition can also be automated, but the rate of ungradable images needs to be closely evaluated. The Automated Retinal Imaging System (ARISTM, Visual Pathways, Inc., Prescott, AZ, USA) [33] and Centervue DRS (Centervue SpA, Padova, Italy) [34] are imaging systems that may be programmed to acquire sequential colour digital images of defined fundus fields using minimally trained technicians. However, these systems have ungradable rates for individual images as high as 30 %. The automated image acquisition in these systems would be helpful in obtaining consistent fields using minimally trained technician; however, this would need to be combined with image quality analysis to determine the need for re-imaging a field. Until much improved automated acquisition systems are available, image pre-processing will likely need to be incorporated into automated image acquisition retinal cameras, allowing real-time assessment of image quality, providing real-time feedback to the technician and allowing a patient to be re-imaged at the same visit if the retinal image does not meet a given standard. Real-time image quality assessment systems are not yet available commercially. One such a system is being developed by VisionQuest Biomedical, LLC, Albuquerque, NM, USA, and has been validated on 2000 colour retinal images showing 100 % sensitivity and 96 % specificity in identifying “rejected” images [35]. Image Analysis An important first step of ARIA for diabetic retinopathy begins with identification and localisation of normal anatomy, better known as “segmentation”, of structures such as the optic nerve head, fovea and large retinal vessels. The main purposes of identifying these structures are to exclude them in the analysis for abnormal lesions and also to determine their locations for image registration (e.g. alignment with a reference image or a previously acquired image) and for use as a reference for
14
Curr Diab Rep (2015) 15:14
Page 4 of 9
measuring distances within the image. Localisation of the optic nerve head is a well-established approach. Some examples of techniques used in this regard include the identification of high pixel intensity within a retinal image (the optic disc is usually the brightest “spot” in a retinal image), principal component analysis (PCA; a method used for face recognition software that differentiates the optic disc from other bright spots such as exudates or reflections in the retina), the Hough transform (which utilises shape orientations to plot locations) and geometrical parametric modelling (a technique that exploits patterns from which retinal vessels emerge from the optic disc) [19, 24]. Automated detection of abnormal diabetic retinopathy lesions was initially performed on fluorescein angiograms [36]. Although fluorescein provides a background with high contrast between vascular and nonvascular structures and allows easier identification of vascular pathology, the requirement of intravenous infusion precluded its wide-scale application. The development of ARIA for diabetic retinopathy has therefore largely focussed on the use of digital colour images of the retina. The main challenge encountered with processing of colour images is the presence of numerous “distractors” within the retinal image (retinal capillaries, underlying choroidal vessels, and reflection artefacts), all of which may be confused with diabetic retinopathy lesions. As a result, much research has been focussed on the selective identification of diabetic retinopathy features, including microaneurysm s, heamorrhages, hard or soft exudates, cotton-wool spots and venous beading. These clinical features have been described in great detail in the landmark clinical trials, Diabetic Retinopathy Study (DRS) and ETDRS. [21, 37]. Earlier work in ARIA utilised basic image processing techniques such as thresholding, edge detection, filters and morphological processing [24]. These techniques focussed on identifying individual diabetic retinopathy features mimicking what is typically performed with manual human grading of individual lesions. Techniques on the extraction of microaneurysms from colour fundus images by the process of thresholding after excluding normal anatomical structures have been extensively reported with varying levels of success [18]. Recent developments in ARIA have moved away from individual detectors to approaches that include the following: an ensemble-based approach (integrates components of microaneurysm detectors) [38], a multiple-lesion approach (fuses multiple classifiers) [39], intelligent systems (which require training algorithms with data derived from human operators) [40] and content-based image retrieval techniques (analyses the contents of large digital image databases and compares it with an archive of labelled images) [41]. The details of algorithms and mathematical modelling methods used and their comparative efficiencies have been published in several reviews [17–19, 24]. These emerging trends of ARIA may prove most relevant to telemedicine given their
potential to go beyond traditional retinal image analysis, possibly providing more detailed assessment of retinopathy severity and greater accuracy of progression risk. Addition of electronic medical record integration may not only allow more accurate prognostication for individual patients but also provide predictive modelling of risk factors based on broad population data.
Overview of ARIA Systems Currently Deployed in Telemedicine and Screening Programmes Current commercially available systems that have been used in telemedicine or screening programmes include the iGradingM (Medalytix Group Ltd), the TRIADTM Network (Hubble Telemedical, Inc.), Iowa Detection Program (IDx, LLC), RetmarkerDR (Retmarker Ltd, Coimbra, Portugal) and Retinalyze System (Retinalyze A/S, Hørsholm, Denmark) (Table 1). A comparison of published sensitivity and specificity of each programme as well as a brief description of the associated published studies are presented in Table 2. Direct head-to-head comparisons between systems have proven difficult, mainly because of different photographic protocols, algorithms and patient populations used for validation. A common thread amongst these automated systems is to identify referable retinopathy: diabetic retinopathy, which requires the attention of an ophthalmologist. To date, we have not progressed to where human intervention can be fully removed from such programmes. All of the systems described below are semi-automated at some point in the workflow pathway and require assistance of a human reader/grader. iGradingM iGradingM is a product of Medalytix Group Ltd, which received its class 1 Conformité Européenne (CE) mark in 2013 and performs “disease/no disease” grading for diabetic retinopathy [42, 43]. It was developed at the University of Aberdeen in Scotland and uses previously published algorithms to assess both image quality and disease and has been described in detail elsewhere [32, 44]. A previously trained automated classifier on a set of 35 images containing 198 individually annotated microaneurysm or dot haemorrhages was used in its development. It was first deployed on a large-scale population in the Scottish Diabetic Retinopathy Screening Program in 2010, after being validated using several large screening populations in Scotland [39, 43, 45•, 46, 47], the largest being 78,601 single-field 45° colour fundus images from 33,535 consecutive patients [45•]. In this retrospective study, 6.6 % of the cohort had referable retinopathy, and iGradingM attained a sensitivity of 97.8 % for referable retinopathy. Although the specificity for referable retinopathy was not reported in the paper, it has since been calculated at 41.1 % [20]. iGradingM
Curr Diab Rep (2015) 15:14 Table 1
Page 5 of 9 14
Overview of current ARIA systems developed for diabetic retinopathy in telemedicine
ARIA system
Company
Developed at
Grading details
Current deployment
iGradingM
Medalytix LLC; Digital Healthcare
University of Aberdeen, Scotland, UK
Diabetic retinopathy absent/ diabetic retinopathy present
The TRIADTM Network
Hubble Telemedicine Inc.
University of Tennessee Health Science Center and the Oak Ridge National Laboratory, USA
IDx-DR
IDx LLC
University of Iowa, USA
RetmarkerDR
Retmarker Ltd.
Coimbra University, Portugal
CBIR technology not approved for use; diagnosis diabetic retinopathy severity with a supervising retinal specialist Diabetic retinopathy index; referable retinopathy (more than mild nonproliferative diabetic retinopathy) Diabetic retinopathy absent/ diabetic retinopathy present; microaneurysm turnover
CE mark 2013 as a class 1 medical device; Scottish Diabetic Retinopathy Screening Program Primary care services, mid-south region of the USA
RetinaLyze A/S
RetinaLyze A/S
Denmark
Microaneurysm and haemorrhage detection
has been further validated on 8271 screening episodes from a South London population in the UK [48]. Here, there was a higher percentage (7.1 %) of referable disease, and a sensitivity of 97.4–99.1 % and specificity of 98.3–99.3 % were attained depending on the configuration used. The TRIAD™ Network The TRIAD™ Network (Telemedical Retinal Image Analysis and Diagnosis) was launched commercially in 2012 by its developers at Hubble Telemedical Inc., and was developed at the University of Tennessee Health Science Center and the Oak Ridge National Laboratory [49]. It consists of a web-based technical network infrastructure that has been functional as a telemedicine network for diabetic retinopathy screening in the mid-south region of the USA since 2009 [50, 51]. ARIA was introduced to the TRIAD™ Network over a period of time, beginning with image quality analysis at the point of image capture [52] and most recently with the use of their patented content-based image retrieval techniques for automated diagnosis [53•]. The technique Table 2
CE mark 2013 as a class IIa medical device
CE mark 2011 as a Class IIa medical device; diabetic retinopathy screening programmes in Portugal CE mark 2011 as a class IIa medical device; software re-launched commercially 2013
of content-based image retrieval involves the comparison of images to large database collections using pictorial content. These image features include various description models, perceptual organisation, spatial relationships and may including clinical metadata [53•]. The authors themselves have reviewed challenges that they have faced with validation of their ARIA software, due mostly to the lack of large, varied and publically available datasets [50]. Briefly, they note that TRIADTM Network infrastructure has created a useful internal validation mechanism by which new algorithms can be developed and tested on datasets from different time points [53•, 54, 55]. Although external validation has been performed for anomaly detection using publically available datasets, these only contain a small number of images (
