Letters RESEARCH LETTER

Outcomes of Adversarial Attacks on Deep Learning Models for Ophthalmology Imaging Domains In the near future, automated systems using deep learning may be used in screening and computer-aided diagnosis in clinical ophthalmology. Recently, researchers have reported subtle adversarial attacks, such as artificial noise created to attack deep learning models, resulting in critical misclassifications.1

Previous dermatology literature has demonstrated that visually imperceptible noise can change a diagnosis from benign to malignant. Because images could be intentionally programmed to force the model to make a mistake, safety concerns are raised for clinical practice. However, while such adversarial attacks have been reported, to our knowledge, they have not been investigated extensively within ophthalmology. This study aims to verify that adversarial attacks can confuse deep learning systems based on imaging domains, such

Figure. Adversarial Attacks on Retinal Images Using InceptionV3 and the Fast Gradient Sign Method A Normal (fundus photography)

Original input image

Generated perturbation

+ε×

ε = 0.050

Result: normal (54.3%)

Result: DR (92.5%)

Result: DR (83.2%)

ε = 0.005

ε = 0.010

ε = 0.050

Result: normal (85.3%)

Result: normal (72.9%)

Result: normal (51.0%)

ε = 0.005

ε = 0.010

ε = 0.050

Result: DR (67.2%)

Result: normal (76.7%)

Result: normal (64.8%)

ε = 0.005

ε = 0.010

ε = 0.050

Result: normal (57.8%)

Result: normal (82.8%)

Result: normal (59.5%)

DR (fundus photography) Original input image

Generated perturbation

+ε×

=

Result: DR (99.9%) C

ε = 0.010

=

Result: normal (100%) B

ε = 0.005

DR (ultrawide-field fundus photography) Original input image

Generated perturbation

+ε×

=

Result: DR (99.9%) D DME (optical coherence tomography)

Original input image

Generated perturbation

+ε×

Result: DME (99.5%)

=

The deep learning models were trained to detect diabetic retinopathy (DR) or diabetic macular edema (DME). A, A normal fundus photograph. B, A fundus photograph with DR. C, An ultrawide-field fundus photograph with DR. D, An optical coherence tomography image with DME.

jamaophthalmology.com

(Reprinted) JAMA Ophthalmology Published online October 1, 2020

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a University of Canberra User on 10/04/2020

E1

Letters

Table. Classification Accuracy of the Conventional Deep Learning Models on Adversarial Examples Crafted by the Fast Gradient Sign Method (FGSM) Using InceptionV3 Accuracy of deep learning models, %a Model

InceptionV3

MobileNetV2

ResNet50

Fundus photography No attack

89.1

88.6

89.9

FGSM using the InceptionV3 modela

13.4

63.7

77.5

Ultrawide-field fundus photography No attack

97.6

97.4

96.8

FGSM using the InceptionV3 modelb

5.0

74.3

72.1

99.6

99.5

99.6

8.2

68.8

64.8

Optical coherence tomography No attack b

FGSM using the InceptionV3 model

The results were derived from the validation dataset.

b

Perturbation coefficient ε = 0.010.

as fundus photography (FP), ultrawide-field FP (UWF), and optical coherence tomography (OCT).

ResNet50, although there were smaller losses of accuracy with these models.

Methods | This study was based on a publicly accessible retinal image database including 35 126 FP images from the Kaggle EyePacs,2 8217 UWF images from the TOP project,3 and 38 208 OCT images from a study by Kermany et al.4 To build binary classifier models to detect diabetic retinopathy or diabetic macular edema, images with other pathologic lesions and ungradable images were excluded from this study. The researchers used open web-based and deidentified data, and this study was determined to be exempt from ethical review according to the Korea National Institute for Bioethics Policy. All procedures were performed in accordance with the ethical standards of the 1964 Helsinki Declaration and its later amendments. The downloaded InceptionV3 deep learning model (Google) was pretrained on the ImageNet database, and the weights of the pretrained networks were fine-tuned for each imaging domain. The input image size was set to a pixel resolution of 224 × 224 by tuning the input tensor. One-tenth of the data set was randomly selected as a validation set. The fast gradient sign method (FGSM) for each imaging domain was used to generate noise for adversarial attack using InceptionV3.5 This is the most popular method of generating adversarial attacks on deep learning using the gradients of the loss function. Google Colab Pro (Google), a cloud service for deep learning research, was used for the experiment. The TensorFlow tutorial page was used to generate FGSM images. All codes are available at https://www.tensorflow.org/tutorials/ generative/adversarial_fgsm.

Discussion | Recently, techniques for adversarial attacks and defenses of deep learning systems have been developed in the artificial intelligence community. Deep learning can produce expert-level diagnoses for diabetic retinopathy; therefore, relatively cheap and fast techniques may replace expensive experts soon. The conventional deep learning models for FP, UWF, and OCT are extremely vulnerable to adversarial attacks, which could be undetectable to humans. These attacks were partially transferable to other deep learning architectures. Because the perturbations were generated using InceptionV3, they were less effective on the MobileNetV2 and ResNet50 models in this study. Our study implies that deep learning may not be the ultimate solution to medical decisionmaking. If medical decisions are performed automatically by deep learning, adversarial attacks can be used for fraudulent purposes.1 Malicious actors could disrupt medical billing and reimbursement systems used by hospitals and insurance companies. Defensive techniques, including training deep learning with adversarial examples, denoising filters, and generative adversarial networks, might be effective at decreasing the effect of adversarial attacks.6 Our results suggest that the designers and approving agencies of medical deep learning systems should be careful to guard against adversarial attacks in clinical ophthalmology.

Results | The FGSM generated FP, UWF, and OCT perturbation images, fooling the deep learning model with very small intensities that would be undetectable by humans (Figure). Images with adversarial attacks can lead the model to misclassify both normal and pathologic input images. Adversarial examples generated using InceptionV3 led the same model to critical misclassification, with accuracy decreasing to 13.4% in FP, 5.0% in UWF, and 8.2% in OCT (Table). The adversarial attacks derived from InceptionV3 were transferable to other conventional deep learning methods, including MobileNetV2 and E2

a

Tae Keun Yoo, MD Joon Yul Choi, PhD Author Affiliations: Aerospace Medical Center, Department of Ophthalmology, Republic of Korea Air Force, Cheongju, South Korea (Yoo); Epilepsy Center, Neurological Institute, Cleveland Clinic, Cleveland, Ohio (Choi). Accepted for Publication: June 30, 2020. Corresponding Author: Tae Keun Yoo, MD, Aerospace Medical Center, Department of Ophthalmology, Republic of Korea Air Force, 635 Danjae-ro, Sangdang-gu, Cheongju, South Korea ([email protected]). Published Online: October 1, 2020. doi:10.1001/jamaophthalmol.2020.3442 Author Contributions: Dr Yoo had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: All authors. Acquisition, analysis, or interpretation of data: All authors.

JAMA Ophthalmology Published online October 1, 2020 (Reprinted)

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a University of Canberra User on 10/04/2020

jamaophthalmology.com

Letters

Drafting of the manuscript: All authors. Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: All authors. Administrative, technical, or material support: All authors. Study supervision: Yoo. Conflict of Interest Disclosures: None reported. 1. Finlayson SG, Bowers JD, Ito J, Zittrain JL, Beam AL, Kohane IS. Adversarial attacks on medical machine learning. Science. 2019;363(6433):1287-1289. doi:10.1126/science.aaw4399 2. Voets M, Møllersen K, Bongo LA. Reproduction study using public data of: development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. PLoS One. 2019;14(6): e0217541. doi:10.1371/journal.pone.0217541

jamaophthalmology.com

3. Nagasawa T, Tabuchi H, Masumoto H, et al. Accuracy of ultrawide-field fundus ophthalmoscopy-assisted deep learning for detecting treatment-naïve proliferative diabetic retinopathy. Int Ophthalmol. 2019;39(10):2153-2159. doi:10.1007/s10792-019-01074-z 4. Kermany DS, Goldbaum M, Cai W, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell. 2018;172(5):1122-1131.e9. doi:10.1016/j.cell.2018.02.010 5. Goodfellow IJ, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. Cornell University Library. December 20, 2014. Accessed April 8, 2020. https://arxiv.org/abs/1412.6572 6. Ren K, Zheng T, Qin Z, Liu X. Adversarial attacks and defenses in deep learning. Eng. 2020;6(3):346-360. doi:10.1016/j.eng.2019.12.012

(Reprinted) JAMA Ophthalmology Published online October 1, 2020

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a University of Canberra User on 10/04/2020

E3

No title

Letters RESEARCH LETTER Outcomes of Adversarial Attacks on Deep Learning Models for Ophthalmology Imaging Domains In the near future, automated syste...
952KB Sizes 0 Downloads 0 Views