Background: Convolutional neural networks (CNNs), have achieved great success in retinal image classification, aiding in the detection of diseases like diabetic retinopathy, cataract, and glaucoma. However, these models are highly vulnerable to gradient-based adversarial attacks, where small perturbations can lead to misclassification. Attacks such as FGSM, PGD, and BIM further demonstrate this vulnerability, raising concerns about their clinical deployment. Developing effective defenses, such as adversarial training, gradient masking, and their ensemble is crucial for ensuring robustness. This study explores these vulnerabilities and defense strategies in the retinal image classification model.

Objectives: The objectives of this study are to evaluate the adversarial robustness of the deep learning model for retinal image classification, particularly under gradient-based attacks like FGSM, PGD, and BIM. The study aims to compare the impact of these attacks on model performance and test the effectiveness of defense mechanisms, including adversarial training, gradient masking, and their ensemble. Additionally, it seeks to identify which defense strategy provides the most reliable protection against adversarial threats in medical imaging.

Methods: A comprehensive literature review is conducted to select the initial models, gradient-based adversarial attacks, and defense mechanisms for evaluation. The methodology involves using ResNet50, DenseNet121, Inception v3, and AlexNetmodels for retinal image classification, evaluated through 5-fold cross-validation to assess their baseline performance. ResNet50, the best-performing model, is subjected to adversarial attacks, including FGSM, PGD, and BIM. The impact of these attacks is measured by comparing performance metrics such as accuracy, precision, recall, and F1-score. Defense mechanisms, including adversarial training, gradient masking, and their ensemble, are then applied to assess their effectiveness in mitigating the attacks and improving model robustness.

Results: The results show that ResNet50 is the best-performing model for retinal image classification, with strong accuracy and efficient inference time. PGD had the most substantial impact on model performance among the attacks tested. Adversarial training effectively reduced the impact of both FGSM and PGD attacks, while gradient masking alone offered a limited defense. The ensemble of adversarial training and gradient masking provided the best protection, significantly improving the model’s robustness across all attacks, including FGSM, PGD, and BIM.

Conclusions: This study demonstrates the critical need for robust defense mechanisms in deep learning models used for retinal image classification. ResNet50 emerged as the most effective model, but its vulnerability to gradient-based attacks highlights the risks in clinical settings. The findings emphasize the importance of developing and implementing effective defenses against adversarial threats in medical imaging.