Cross-Dataset Generalization Analysis of Deep Transfer Learning Models for Diabetic Retinopathy Classification

AUTHORS

1. Argina Akter

2. Abid Shahriar

3. M. Chowdhury

   Lamar University, Beaumont, Texas, USA

4. Adnan Shafi

   Lamar University, Beaumont, Texas, USA

5. T. Rahman

   Lamar University, Beaumont, Texas, USA

6. A. Miraz

   Lamar University, Beaumont, Texas, USA

7. Shakeef Ahmed Rakin

   Department of Computer Science and Engineering, BRAC University Dhaka, Bangladesh

8. Mursalin Leon

9. Atz Prince

10. Dipta Pal

ABSTRACT

Deep learning has demonstrated strong potential for automated diabetic retinopathy (DR) detection. However, many high performing models fail to maintain reliability when tested on unseen clinical datasets. This research presents a comparative evaluation of four transfer learning architectures EfficientNetB2, ResNet50, DenseNet121, and VGG16 for multi-class DR classification with emphasis on cross-dataset generalization. The models were trained using the APTOS 2019 retinal image dataset and externally evaluated on the Messidor-2 dataset to examine robustness against domain shift caused by differences in imaging devices, illumination, and patient populations. A unified preprocessing and training pipeline was adopted to ensure fair comparison. Performance was assessed using accuracy, precision, recall, F1-score, and generalization drop percentage. Experimental findings indicate that all models experienced measurable performance decline on external data, confirming the presence of a generalization gap. Among the evaluated architectures, EfficientNetB2 achieved the most stable overall performance with the lowest generalization drop and competitive classification metrics. These results highlight the necessity of external validation for selecting dependable artificial intelligence models for practical DR screening applications.