-
Beyond Natural-Image Foundation Models: Benchmarking Satellite Pretraining for Ophthalmic Image Analysis
Authors:
Lovre Antonio Budimir,
Mingya Alexa Gong,
Alyssa Foong Quinney,
Ivana Matovinović,
Yukun Zhou,
Pearse A. Keane,
Sven Lončarić,
Marinko V. Šarunić
Abstract:
Vision Foundation Models (VFMs) have emerged as a promising approach in medical imaging, producing broadly applicable systems that can be efficiently adapted across diverse imaging modalities, anatomical regions, and clinical tasks. However, VFMs require extensive training data, and their progress in medical image analysis is constrained by limited data availability, privacy concerns, and high dev…
▽ More
Vision Foundation Models (VFMs) have emerged as a promising approach in medical imaging, producing broadly applicable systems that can be efficiently adapted across diverse imaging modalities, anatomical regions, and clinical tasks. However, VFMs require extensive training data, and their progress in medical image analysis is constrained by limited data availability, privacy concerns, and high development costs. To alleviate these constraints, medical VFMs (MedVFMs) are often built upon weights from generalist models pretrained on vast amounts of publicly available natural images, introducing a substantial distribution shift for medical task adaptation. To address this, we propose satellite imagery as a novel pretraining domain for MedVFM development and benchmarking, motivated by its closer visual alignment with medical data and its freedom from the privacy constraints that limit medical datasets. Across multiple ophthalmic imaging modalities, we compare DINOv3-SAT493m pretrained on 493 million satellite images against DINOv3-LVD1689m pretrained on 1.7 billion natural images, together with two medical specialist baselines: DINOv3-RETFound and MAE-RETFound. Our experiments show that satellite imagery is a stronger pretraining source than natural images for ophthalmic tasks, particularly on en face vascular-rich modalities. On several tasks, satellite pretraining matches or exceeds the medical specialists on high-resolution en face inputs, despite using no medical data.
△ Less
Submitted 15 August, 2026;
originally announced August 2026.
-
Representation Transfer of Foundation Models for Ultra-Widefield Retinal Imaging
Authors:
Mingya Alexa Gong,
Da Ma,
Lovre Antonio Budimir,
Ivana Matovinovic,
Sven Loncaric,
Myeong Jin Ju,
Yukun Zhou,
Siegfried K. Wagner,
Pearse A. Keane,
Marinko V. Sarunic
Abstract:
Despite the widespread adoption of foundation models as feature extractors for medical imaging, relatively little is understood about how different pretraining strategies influence the transferability of learned representations to weakly supervised ophthalmic imaging tasks. We investigate this question in ultra-widefield (UWF) retinal imaging by evaluating foundation model representations within a…
▽ More
Despite the widespread adoption of foundation models as feature extractors for medical imaging, relatively little is understood about how different pretraining strategies influence the transferability of learned representations to weakly supervised ophthalmic imaging tasks. We investigate this question in ultra-widefield (UWF) retinal imaging by evaluating foundation model representations within a patch-based multiple instance learning (MIL) framework for disease classification on UWF images. We compare Vision Transformer encoders pretrained with supervised, Masked Autoencoder (MAE), and self-distillation objectives, while keeping the downstream aggregation architecture unchanged. Within a controlled comparison of ViT-B encoders pretrained on ImageNet-1k, the choice of pretraining objective substantially influenced frozen representation transfer, with supervised and self-distillation-based models outperforming MAE. A contemporary DINOv3 model pretrained at a larger scale achieved the strongest overall performance, with a quadratic weighted kappa of 0.863 for five-class diabetic retinopathy grading, comparable with DINOv1. Attention analysis further revealed distinct patch-aggregation behaviours associated with the different pretrained representations, while partial fine-tuning substantially reduced the performance gap for MAE. These findings suggest that pretraining strategy influences both representation transferability and the subsequent aggregation of patch-level evidence within MIL, resulting in differences in downstream classification performance.
△ Less
Submitted 1 August, 2026;
originally announced August 2026.
-
Deep Learning for Retinal Degeneration Assessment: A Comprehensive Analysis of the MARIO Challenge
Authors:
Rachid Zeghlache,
Ikram Brahim,
Pierre-Henri Conze,
Mathieu Lamard,
Mohammed El Amine Lazouni,
Zineb Aziza Elaouaber,
Leila Ryma Lazouni,
Christopher Nielsen,
Ahmad O. Ahsan,
Matthias Wilms,
Nils D. Forkert,
Lovre Antonio Budimir,
Ivana Matovinović,
Donik Vršnak,
Sven Lončarić,
Philippe Zhang,
Weili Jiang,
Yihao Li,
Yiding Hao,
Markus Frohmann,
Patrick Binder,
Marcel Huber,
Taha Emre,
Teresa Finisterra Araújo,
Marzieh Oghbaie
, et al. (25 additional authors not shown)
Abstract:
The MARIO challenge, held at MICCAI 2024, focused on advancing the automated detection and monitoring of age-related macular degeneration (AMD) through the analysis of optical coherence tomography (OCT) images. Designed to evaluate algorithmic performance in detecting neovascular activity changes within AMD, the challenge incorporated unique multi-modal datasets. The primary dataset, sourced from…
▽ More
The MARIO challenge, held at MICCAI 2024, focused on advancing the automated detection and monitoring of age-related macular degeneration (AMD) through the analysis of optical coherence tomography (OCT) images. Designed to evaluate algorithmic performance in detecting neovascular activity changes within AMD, the challenge incorporated unique multi-modal datasets. The primary dataset, sourced from Brest, France, was used by participating teams to train and test their models. The final ranking was determined based on performance on this dataset. An auxiliary dataset from Algeria was used post-challenge to evaluate population and device shifts from submitted solutions. Two tasks were involved in the MARIO challenge. The first one was the classification of evolution between two consecutive 2D OCT B-scans. The second one was the prediction of future AMD evolution over three months for patients undergoing anti-vascular endothelial growth factor (VEGF) therapy. Thirty-five teams participated, with the top 12 finalists presenting their methods. This paper outlines the challenge's structure, tasks, data characteristics, and winning methodologies, setting a benchmark for AMD monitoring using OCT, infrared imaging, and clinical data (such as the number of visits, age, gender, etc.). The results of this challenge indicate that artificial intelligence (AI) performs as well as a physician in measuring AMD progression (Task 1) but is not yet able of predicting future evolution (Task 2).
△ Less
Submitted 3 August, 2026; v1 submitted 3 June, 2025;
originally announced June 2025.
-
Microvasculature Segmentation and Inter-capillary Area Quantification of the Deep Vascular Complex using Transfer Learning
Authors:
Julian Lo,
Morgan Heisler,
Vinicius Vanzan,
Sonja Karst,
Ivana Zadro Matovinovic,
Sven Loncaric,
Eduardo V. Navajas,
Mirza Faisal Beg,
Marinko V. Sarunic
Abstract:
Purpose: Optical Coherence Tomography Angiography (OCT-A) permits visualization of the changes to the retinal circulation due to diabetic retinopathy (DR), a microvascular complication of diabetes. We demonstrate accurate segmentation of the vascular morphology for the superficial capillary plexus and deep vascular complex (SCP and DVC) using a convolutional neural network (CNN) for quantitative a…
▽ More
Purpose: Optical Coherence Tomography Angiography (OCT-A) permits visualization of the changes to the retinal circulation due to diabetic retinopathy (DR), a microvascular complication of diabetes. We demonstrate accurate segmentation of the vascular morphology for the superficial capillary plexus and deep vascular complex (SCP and DVC) using a convolutional neural network (CNN) for quantitative analysis.
Methods: Retinal OCT-A with a 6x6mm field of view (FOV) were acquired using a Zeiss PlexElite. Multiple-volume acquisition and averaging enhanced the vessel network contrast used for training the CNN. We used transfer learning from a CNN trained on 76 images from smaller FOVs of the SCP acquired using different OCT systems. Quantitative analysis of perfusion was performed on the automated vessel segmentations in representative patients with DR.
Results: The automated segmentations of the OCT-A images maintained the hierarchical branching and lobular morphologies of the SCP and DVC, respectively. The network segmented the SCP with an accuracy of 0.8599, and a Dice index of 0.8618. For the DVC, the accuracy was 0.7986, and the Dice index was 0.8139. The inter-rater comparisons for the SCP had an accuracy and Dice index of 0.8300 and 0.6700, respectively, and 0.6874 and 0.7416 for the DVC.
Conclusions: Transfer learning reduces the amount of manually-annotated images required, while producing high quality automatic segmentations of the SCP and DVC. Using high quality training data preserves the characteristic appearance of the capillary networks in each layer.
Translational Relevance: Accurate retinal microvasculature segmentation with the CNN results in improved perfusion analysis in diabetic retinopathy.
△ Less
Submitted 19 March, 2020;
originally announced March 2020.