-
Uncovering Overconfident Failures in CXR Models via Augmentation-Sensitivity Risk Scoring
Authors:
Han-Jay Shu,
Wei-Ning Chiu,
Shun-Ting Chang,
Meng-Ping Huang,
Takeshi Tohyama,
Ahram Han,
Po-Chih Kuo
Abstract:
Deep learning models achieve strong performance in chest radiograph (CXR) interpretation, yet fairness and reliability concerns persist. Models often show uneven accuracy across patient subgroups, leading to hidden failures not reflected in aggregate metrics. Existing error detection approaches -- based on confidence calibration or out-of-distribution (OOD) detection -- struggle with subtle within…
▽ More
Deep learning models achieve strong performance in chest radiograph (CXR) interpretation, yet fairness and reliability concerns persist. Models often show uneven accuracy across patient subgroups, leading to hidden failures not reflected in aggregate metrics. Existing error detection approaches -- based on confidence calibration or out-of-distribution (OOD) detection -- struggle with subtle within-distribution errors, while image- and representation-level consistency-based methods remain underexplored in medical imaging. We propose an augmentation-sensitivity risk scoring (ASRS) framework to identify error-prone CXR cases. ASRS applies clinically plausible rotations ($\pm 15^\circ$/$\pm 30^\circ$) and measures embedding shifts with the RAD-DINO encoder. Sensitivity scores stratify samples into stability quartiles, where highly sensitive cases show substantially lower recall ($-0.2$ to $-0.3$) despite high AUROC and confidence. ASRS provides a label-free means for selective prediction and clinician review, improving fairness and safety in medical AI.
△ Less
Submitted 2 October, 2025;
originally announced October 2025.
-
Red Teaming for Generative AI, Report on a Copyright-Focused Exercise Completed in an Academic Medical Center
Authors:
James Wen,
Sahil Nalawade,
Zhiwei Liang,
Catherine Bielick,
Marisa Ferrara Boston,
Alexander Chowdhury,
Adele Collin,
Luigi De Angelis,
Jacob Ellen,
Heather Frase,
Rodrigo R. Gameiro,
Juan Manuel Gutierrez,
Pooja Kadam,
Murat Keceli,
Srikanth Krishnamurthy,
Anne Kwok,
Yanan Lance Lu,
Heather Mattie,
Liam G. McCoy,
Katherine Miller,
Allison C. Morgan,
Marlene Louisa Moerig,
Trang Nguyen,
Alexander Owen-Post,
Alex D. Ruiz
, et al. (16 additional authors not shown)
Abstract:
Background: Generative artificial intelligence (AI) deployment in academic medical settings raises copyright compliance concerns. Dana-Farber Cancer Institute implemented GPT4DFCI, an internal generative AI tool utilizing OpenAI models, that is approved for enterprise use in research and operations. Given (1) the exceptionally broad adoption of the tool in our organization, (2) our research missio…
▽ More
Background: Generative artificial intelligence (AI) deployment in academic medical settings raises copyright compliance concerns. Dana-Farber Cancer Institute implemented GPT4DFCI, an internal generative AI tool utilizing OpenAI models, that is approved for enterprise use in research and operations. Given (1) the exceptionally broad adoption of the tool in our organization, (2) our research mission, and (3) the shared responsibility model required to benefit from Customer Copyright Commitment in Azure OpenAI Service products, we deemed rigorous copyright compliance testing necessary.
Case Description: We conducted a structured red teaming exercise in Nov. 2024, with 42 participants from academic, industry, and government institutions. Four teams attempted to extract copyrighted content from GPT4DFCI across four domains: literary works, news articles, scientific publications, and access-restricted clinical notes. Teams successfully extracted verbatim book dedications and near-exact passages through various strategies. News article extraction failed despite jailbreak attempts. Scientific article reproduction yielded only high-level summaries. Clinical note testing revealed appropriate privacy safeguards.
Discussion: The successful extraction of literary content indicates potential copyrighted material presence in training data, necessitating inference-time filtering. Differential success rates across content types suggest varying protective mechanisms. The event led to implementation of a copyright-specific meta-prompt in GPT4DFCI; this mitigation has been in production since Jan. 2025.
Conclusion: Systematic red teaming revealed specific vulnerabilities in generative AI copyright compliance, leading to concrete mitigation strategies. Academic medical institutions deploying generative AI should implement continuous testing protocols to ensure legal and ethical compliance.
△ Less
Submitted 2 July, 2025; v1 submitted 26 June, 2025;
originally announced June 2025.
-
WorldMedQA-V: a multilingual, multimodal medical examination dataset for multimodal language models evaluation
Authors:
João Matos,
Shan Chen,
Siena Placino,
Yingya Li,
Juan Carlos Climent Pardo,
Daphna Idan,
Takeshi Tohyama,
David Restrepo,
Luis F. Nakayama,
Jose M. M. Pascual-Leone,
Guergana Savova,
Hugo Aerts,
Leo A. Celi,
A. Ian Wong,
Danielle S. Bitterman,
Jack Gallifant
Abstract:
Multimodal/vision language models (VLMs) are increasingly being deployed in healthcare settings worldwide, necessitating robust benchmarks to ensure their safety, efficacy, and fairness. Multiple-choice question and answer (QA) datasets derived from national medical examinations have long served as valuable evaluation tools, but existing datasets are largely text-only and available in a limited su…
▽ More
Multimodal/vision language models (VLMs) are increasingly being deployed in healthcare settings worldwide, necessitating robust benchmarks to ensure their safety, efficacy, and fairness. Multiple-choice question and answer (QA) datasets derived from national medical examinations have long served as valuable evaluation tools, but existing datasets are largely text-only and available in a limited subset of languages and countries. To address these challenges, we present WorldMedQA-V, an updated multilingual, multimodal benchmarking dataset designed to evaluate VLMs in healthcare. WorldMedQA-V includes 568 labeled multiple-choice QAs paired with 568 medical images from four countries (Brazil, Israel, Japan, and Spain), covering original languages and validated English translations by native clinicians, respectively. Baseline performance for common open- and closed-source models are provided in the local language and English translations, and with and without images provided to the model. The WorldMedQA-V benchmark aims to better match AI systems to the diverse healthcare environments in which they are deployed, fostering more equitable, effective, and representative applications.
△ Less
Submitted 16 October, 2024;
originally announced October 2024.