-
FusionFM: Fusing Eye-specific Foundational Models for Optimized Ophthalmic Diagnosis
Authors:
Ke Zou,
Jocelyn Hui Lin Goh,
Yukun Zhou,
Tian Lin,
Samantha Min Er Yew,
Sahana Srinivasan,
Meng Wang,
Rui Santos,
Gabor M. Somfai,
Huazhu Fu,
Haoyu Chen,
Pearse A. Keane,
Ching-Yu Cheng,
Yih Chung Tham
Abstract:
Foundation models (FMs) have shown great promise in medical image analysis by improving generalization across diverse downstream tasks. In ophthalmology, several FMs have recently emerged, but there is still no clear answer to fundamental questions: Which FM performs the best? Are they equally good across different tasks? What if we combine all FMs together? To our knowledge, this is the first stu…
▽ More
Foundation models (FMs) have shown great promise in medical image analysis by improving generalization across diverse downstream tasks. In ophthalmology, several FMs have recently emerged, but there is still no clear answer to fundamental questions: Which FM performs the best? Are they equally good across different tasks? What if we combine all FMs together? To our knowledge, this is the first study to systematically evaluate both single and fused ophthalmic FMs. To address these questions, we propose FusionFM, a comprehensive evaluation suite, along with two fusion approaches to integrate different ophthalmic FMs. Our framework covers both ophthalmic disease detection (glaucoma, diabetic retinopathy, and age-related macular degeneration) and systemic disease prediction (diabetes and hypertension) based on retinal imaging. We benchmarked four state-of-the-art FMs (RETFound, VisionFM, RetiZero, and DINORET) using standardized datasets from multiple countries and evaluated their performance using AUC and F1 metrics. Our results show that DINORET and RetiZero achieve superior performance in both ophthalmic and systemic disease tasks, with RetiZero exhibiting stronger generalization on external datasets. Regarding fusion strategies, the Gating-based approach provides modest improvements in predicting glaucoma, AMD, and hypertension. Despite these advances, predicting systemic diseases, especially hypertension in external cohort remains challenging. These findings provide an evidence-based evaluation of ophthalmic FMs, highlight the benefits of model fusion, and point to strategies for enhancing their clinical applicability.
△ Less
Submitted 14 August, 2025;
originally announced August 2025.
-
Is an Ultra Large Natural Image-Based Foundation Model Superior to a Retina-Specific Model for Detecting Ocular and Systemic Diseases?
Authors:
Qingshan Hou,
Yukun Zhou,
Jocelyn Hui Lin Goh,
Ke Zou,
Samantha Min Er Yew,
Sahana Srinivasan,
Meng Wang,
Thaddaeus Lo,
Xiaofeng Lei,
Siegfried K. Wagner,
Mark A. Chia,
Dawei Yang,
Hongyang Jiang,
An Ran Ran,
Rui Santos,
Gabor Mark Somfai,
Juan Helen Zhou,
Haoyu Chen,
Qingyu Chen,
Carol Y. Cheung,
Pearse A. Keane,
Yih Chung Tham
Abstract:
The advent of foundation models (FMs) is transforming medical domain. In ophthalmology, RETFound, a retina-specific FM pre-trained sequentially on 1.4 million natural images and 1.6 million retinal images, has demonstrated high adaptability across clinical applications. Conversely, DINOv2, a general-purpose vision FM pre-trained on 142 million natural images, has shown promise in non-medical domai…
▽ More
The advent of foundation models (FMs) is transforming medical domain. In ophthalmology, RETFound, a retina-specific FM pre-trained sequentially on 1.4 million natural images and 1.6 million retinal images, has demonstrated high adaptability across clinical applications. Conversely, DINOv2, a general-purpose vision FM pre-trained on 142 million natural images, has shown promise in non-medical domains. However, its applicability to clinical tasks remains underexplored. To address this, we conducted head-to-head evaluations by fine-tuning RETFound and three DINOv2 models (large, base, small) for ocular disease detection and systemic disease prediction tasks, across eight standardized open-source ocular datasets, as well as the Moorfields AlzEye and the UK Biobank datasets. DINOv2-large model outperformed RETFound in detecting diabetic retinopathy (AUROC=0.850-0.952 vs 0.823-0.944, across three datasets, all P<=0.007) and multi-class eye diseases (AUROC=0.892 vs. 0.846, P<0.001). In glaucoma, DINOv2-base model outperformed RETFound (AUROC=0.958 vs 0.940, P<0.001). Conversely, RETFound achieved superior performance over all DINOv2 models in predicting heart failure, myocardial infarction, and ischaemic stroke (AUROC=0.732-0.796 vs 0.663-0.771, all P<0.001). These trends persisted even with 10% of the fine-tuning data. These findings showcase the distinct scenarios where general-purpose and domain-specific FMs excel, highlighting the importance of aligning FM selection with task-specific requirements to optimise clinical performance.
△ Less
Submitted 4 September, 2025; v1 submitted 10 February, 2025;
originally announced February 2025.
-
Are Traditional Deep Learning Model Approaches as Effective as a Retinal-Specific Foundation Model for Ocular and Systemic Disease Detection?
Authors:
Samantha Min Er Yew,
Xiaofeng Lei,
Jocelyn Hui Lin Goh,
Yibing Chen,
Sahana Srinivasan,
Miao-li Chee,
Krithi Pushpanathan,
Ke Zou,
Qingshan Hou,
Zhi Da Soh,
Cancan Xue,
Marco Chak Yan Yu,
Charumathi Sabanayagam,
E Shyong Tai,
Xueling Sim,
Yaxing Wang,
Jost B. Jonas,
Vinay Nangia,
Gabriel Dawei Yang,
Emma Anran Ran,
Carol Yim-Lui Cheung,
Yangqin Feng,
Jun Zhou,
Rick Siow Mong Goh,
Yukun Zhou
, et al. (4 additional authors not shown)
Abstract:
Background: RETFound, a self-supervised, retina-specific foundation model (FM), showed potential in downstream applications. However, its comparative performance with traditional deep learning (DL) models remains incompletely understood. This study aimed to evaluate RETFound against three ImageNet-pretrained supervised DL models (ResNet50, ViT-base, SwinV2) in detecting ocular and systemic disease…
▽ More
Background: RETFound, a self-supervised, retina-specific foundation model (FM), showed potential in downstream applications. However, its comparative performance with traditional deep learning (DL) models remains incompletely understood. This study aimed to evaluate RETFound against three ImageNet-pretrained supervised DL models (ResNet50, ViT-base, SwinV2) in detecting ocular and systemic diseases.
Methods: We fine-tuned/trained RETFound and three DL models on full datasets, 50%, 20%, and fixed sample sizes (400, 200, 100 images, with half comprising disease cases; for each DR severity class, 100 and 50 cases were used. Fine-tuned models were tested internally using the SEED (53,090 images) and APTOS-2019 (3,672 images) datasets and externally validated on population-based (BES, CIEMS, SP2, UKBB) and open-source datasets (ODIR-5k, PAPILA, GAMMA, IDRiD, MESSIDOR-2). Model performance was compared using area under the receiver operating characteristic curve (AUC) and Z-tests with Bonferroni correction (P<0.05/3).
Interpretation: Traditional DL models are mostly comparable to RETFound for ocular disease detection with large datasets. However, RETFound is superior in systemic disease detection with smaller datasets. These findings offer valuable insights into the respective merits and limitation of traditional models and FMs.
△ Less
Submitted 21 January, 2025;
originally announced January 2025.
-
RG-CAT: Detection Pipeline and Catalogue of Radio Galaxies in the EMU Pilot Survey
Authors:
Nikhel Gupta,
Ray P. Norris,
Zeeshan Hayder,
Minh Huynh,
Lars Petersson,
X. Rosalind Wang,
Andrew M. Hopkins,
Heinz Andernach,
Yjan Gordon,
Simone Riggi,
Miranda Yew,
Evan J. Crawford,
Bärbel Koribalski,
Miroslav D. Filipović,
Anna D. Kapinśka,
Stanislav Shabala,
Tessa Vernstrom,
Joshua R. Marvil
Abstract:
We present source detection and catalogue construction pipelines to build the first catalogue of radio galaxies from the 270 $\rm deg^2$ pilot survey of the Evolutionary Map of the Universe (EMU-PS) conducted with the Australian Square Kilometre Array Pathfinder (ASKAP) telescope. The detection pipeline uses Gal-DINO computer-vision networks (Gupta et al., 2024) to predict the categories of radio…
▽ More
We present source detection and catalogue construction pipelines to build the first catalogue of radio galaxies from the 270 $\rm deg^2$ pilot survey of the Evolutionary Map of the Universe (EMU-PS) conducted with the Australian Square Kilometre Array Pathfinder (ASKAP) telescope. The detection pipeline uses Gal-DINO computer-vision networks (Gupta et al., 2024) to predict the categories of radio morphology and bounding boxes for radio sources, as well as their potential infrared host positions. The Gal-DINO network is trained and evaluated on approximately 5,000 visually inspected radio galaxies and their infrared hosts, encompassing both compact and extended radio morphologies. We find that the Intersection over Union (IoU) for the predicted and ground truth bounding boxes is larger than 0.5 for 99% of the radio sources, and 98% of predicted host positions are within $3^{\prime \prime}$ of the ground truth infrared host in the evaluation set. The catalogue construction pipeline uses the predictions of the trained network on the radio and infrared image cutouts based on the catalogue of radio components identified using the Selavy source finder algorithm. Confidence scores of the predictions are then used to prioritize Selavy components with higher scores and incorporate them first into the catalogue. This results in identifications for a total of 211,625 radio sources, with 201,211 classified as compact and unresolved. The remaining 10,414 are categorized as extended radio morphologies, including 582 FR-I, 5,602 FR-II, 1,494 FR-x (uncertain whether FR-I or FR-II), 2,375 R (single-peak resolved) radio galaxies, and 361 with peculiar and other rare morphologies. We cross-match the radio sources in the catalogue with the infrared and optical catalogues, finding infrared cross-matches for 73% and photometric redshifts for 36% of the radio galaxies.
△ Less
Submitted 21 March, 2024;
originally announced March 2024.
-
Deep Learning for Morphological Identification of Extended Radio Galaxies using Weak Labels
Authors:
Nikhel Gupta,
Zeeshan Hayder,
Ray P. Norris,
Minh Huynh,
Lars Petersson,
X. Rosalind Wang,
Heinz Andernach,
Bärbel S. Koribalski,
Miranda Yew,
Evan J. Crawford
Abstract:
The present work discusses the use of a weakly-supervised deep learning algorithm that reduces the cost of labelling pixel-level masks for complex radio galaxies with multiple components. The algorithm is trained on weak class-level labels of radio galaxies to get class activation maps (CAMs). The CAMs are further refined using an inter-pixel relations network (IRNet) to get instance segmentation…
▽ More
The present work discusses the use of a weakly-supervised deep learning algorithm that reduces the cost of labelling pixel-level masks for complex radio galaxies with multiple components. The algorithm is trained on weak class-level labels of radio galaxies to get class activation maps (CAMs). The CAMs are further refined using an inter-pixel relations network (IRNet) to get instance segmentation masks over radio galaxies and the positions of their infrared hosts. We use data from the Australian Square Kilometre Array Pathfinder (ASKAP) telescope, specifically the Evolutionary Map of the Universe (EMU) Pilot Survey, which covered a sky area of 270 square degrees with an RMS sensitivity of 25-35 $μ$Jy/beam. We demonstrate that weakly-supervised deep learning algorithms can achieve high accuracy in predicting pixel-level information, including masks for the extended radio emission encapsulating all galaxy components and the positions of the infrared host galaxies. We evaluate the performance of our method using mean Average Precision (mAP) across multiple classes at a standard intersection over union (IoU) threshold of 0.5. We show that the model achieves a mAP$_{50}$ of 67.5\% and 76.8\% for radio masks and infrared host positions, respectively. The network architecture can be found at the following link: https://github.com/Nikhel1/Gal-CAM
△ Less
Submitted 9 August, 2023;
originally announced August 2023.
-
An SME's Adoption of a Cloud Based Integrated Management System (IMS) When Certifying Against Management System Standards (MSS)
Authors:
Ming Hock Yew,
Jenson Goh
Abstract:
This case study introduces a four step approach used by a Singapore small and medium enterprise (SME) in implementing a cloud computing based integrated management system (IMS) to meet the ISO 9001, ISO 14001, and OHSAS 18001 certification requirements. The objectives of this case study are to study: (1) the challenges encountered by an SME during the IMS integration process at each of the four le…
▽ More
This case study introduces a four step approach used by a Singapore small and medium enterprise (SME) in implementing a cloud computing based integrated management system (IMS) to meet the ISO 9001, ISO 14001, and OHSAS 18001 certification requirements. The objectives of this case study are to study: (1) the challenges encountered by an SME during the IMS integration process at each of the four levels of integration (2) the extent which Frugal IT Innovation and Technology Acceptance Model (TAM3) concepts apply to our four step approach. The four step approach was assessed against a framework of four integration levels of the management systems (MS), and also the applicability of two IS theories (Frugal IT Innovation and TAM3). Data was collected via: (1) direct observations (2) participant observations; (3) interviews with key personnel involved in the project; and (4) analysis of documents pertaining to the project. This case study provides an exemplary model on using IS theories and public cloud technologies to develop an effective IMS that other SMEs can learn from.
△ Less
Submitted 10 June, 2016;
originally announced June 2016.