Explainable multimodal foundational models for retinal disease stratification: a robustness study across 15+ heterogeneous datasets

Vista sencilla de ítem
dc.contributor.authorOsa Sánchez, Ainhoa
dc.contributor.authorEl-Baz, Ayman
dc.contributor.authorOleagordia Ruiz, Ibon
dc.contributor.authorGarcía-Zapirain, Begoña
dc.date.accessioned2026-04-13T14:59:46Z
dc.date.available2026-04-13T14:59:46Z
dc.date.issued2026-02-25
dc.date.updated2026-04-13T14:59:46Z
dc.description.abstractThe automated stratification of retinal diseases remains a significant challenge due to data heterogeneity and the closed-box nature of deep learning models. Although foundational models have demonstrated remarkable success in general computer vision, their clinical reliability and interpretability in multimodal ophthalmology remain insufficiently explored. In this work, we introduce an Explainable Multimodal Foundational AI framework trained on a large-scale integrated corpus of 760,243 retinal images collected from over 15 heterogeneous repositories, encompassing both fundus photography and optical coherence tomography (OCT). We systematically evaluate self-supervised learning (SSL) paradigms DINO and iBOT across convolutional (ResNet) and Transformer-based (Vision Transformer, ViT) architectures. Our results show that ResNet-DINO achieves state-of-the-art performance, reaching 93.53% accuracy and a 0.935 F1-score in 6-class multimodal retinal disease classification, while exhibiting superior robustness under data-limited conditions, attributed to its inductive bias. Notably, we observe emergent clinical localization capabilities in Vision Transformer models (ViT-DINOv2 and ViT-iBOT). Using frozen pre-trained weights and without exposure to expert-labeled data or ground truth labels, these models autonomously highlight clinically relevant biomarkers, including subretinal fluid and drusen, demonstrating intrinsic pathological awareness. By bridging the semantic gap between unsupervised representation learning and targeted clinical diagnosis, this study establishes a benchmark for robust, explainable, and label-efficient AI in ophthalmology. Our findings indicate that large-scale foundational pre-training not only enhances diagnostic accuracy but also induces meaningful visual priors aligned with established clinical biomarkers, supporting the deployment of trustworthy AI systems in real-world clinical decision support.en
dc.description.sponsorshipThis work was supported by the Basque Government through the Hazitek 2024 program, Spain, within the framework of the IRUD-IA project: ‘‘Medical Image Analysis Technologies with Artificial Intelligence for the Development of Medical Devices,’’ project code ZE-2024/00030en
dc.identifier.citationOsa-Sanchez, A., El-Baz, A., Oleagordia-Ruiz, I., & Garcia-Zapirain, B. (2026). Explainable multimodal foundational models for retinal disease stratification: a robustness study across 15+ heterogeneous datasets. IEEE Access, 14, 31567-31579. https://doi.org/10.1109/ACCESS.2026.3668034
dc.identifier.doi10.1109/ACCESS.2026.3668034
dc.identifier.eissn2169-3536
dc.identifier.urihttps://hdl.handle.net/20.500.14454/5627
dc.language.isoeng
dc.publisherInstitute of Electrical and Electronics Engineers Inc.
dc.rights© 2026 The Authors
dc.subject.otherExplainable AI (XAI)
dc.subject.otherFoundation models
dc.subject.otherLarge-scale ophthalmic benchmark
dc.subject.otherMultimodal fusion
dc.subject.otherRetinal pathology
dc.subject.otherSelf-supervised learning
dc.titleExplainable multimodal foundational models for retinal disease stratification: a robustness study across 15+ heterogeneous datasetsen
dc.typejournal article
dcterms.accessRightsopen access
oaire.citation.endPage31579
oaire.citation.startPage31567
oaire.citation.titleIEEE Access
oaire.citation.volume14
oaire.licenseConditionhttps://creativecommons.org/licenses/by/4.0/
oaire.versionVoR
Archivos
Bloque original
Mostrando 1 - 1 de 1
Miniatura
Nombre:
osa_explainable_2026.pdf
Tamaño:
1.89 MB
Formato:
Adobe Portable Document Format
Descargar
Colecciones
Artículos