-
Benchmarking and Evaluating VLMs for Software Architecture Diagram Understanding
Authors:
Shuyin Ouyang,
Jie M. Zhang,
Jingzhi Gong,
Gunel Jahangirova,
Mohammad Reza Mousavi,
Jack Johns,
Beum Seuk Lee,
Adam Ziolkowski,
Botond Virginas,
Joost Noppen
Abstract:
Software architecture diagrams are important design artifacts for communicating system structure, behavior, and data organization throughout the software development lifecycle. Although recent progress in large language models has substantially advanced code-centric software engineering tasks such as code generation, testing, and maintenance, the ability of modern vision-language models (VLMs) to…
▽ More
Software architecture diagrams are important design artifacts for communicating system structure, behavior, and data organization throughout the software development lifecycle. Although recent progress in large language models has substantially advanced code-centric software engineering tasks such as code generation, testing, and maintenance, the ability of modern vision-language models (VLMs) to understand software architecture diagrams remains underexplored. To address this gap, we present SADU, a benchmark for Software Architecture Diagram Understanding that evaluates VLMs on architecture diagrams as structured software engineering artifacts rather than generic images. SADU contains 154 carefully curated diagrams spanning behavioral, structural, and ER diagrams, paired with structured annotations and 2,431 question-answer tasks covering counting and retrieval reasoning. We evaluate 11 state-of-the-art VLMs from the Gemini, Claude, GPT, and Qwen families.
Our results show that software architecture diagram understanding remains challenging for current models: the best-performing model gemini-3-flash-preview achieves only 70.18\% accuracy, while gpt-4o-mini only achieves 17.77\% accuracy. The results further reveal the weaknesses in diagram reasoning and visual relation grounding, highlighting a gap between current VLMs and the needs of design-stage software engineering. SADU provides a foundation for future research on diagram-aware AI systems and more faithful AI-assisted software engineering workflows.
△ Less
Submitted 5 April, 2026;
originally announced April 2026.
-
The UK and Ireland Geophysical Array -- Concept and Design
Authors:
Andrew Curtis,
Karen Lythgoe,
Stephen P. Hicks,
Lidong Bie,
Dominik Strutz,
Emma Chambers,
Brian Baptie,
Dave Cornwell,
Juliane Huebert,
Jessica Irving,
Glenn Jones,
Sergei Lebedev,
Walid Ben Mansour,
Aideliz Montiel Álvarez,
Stuart Nippress,
Koen Van Noten,
Tim Pharaoh,
Romesh Palamakumbura,
Nick Rawlinson,
Pablo Rodriguez Salgado,
James Verdon,
Chuanbin Zhu,
Wen Zhou,
Jelle Assink,
Ian Bastow
, et al. (41 additional authors not shown)
Abstract:
Scientific exploration of the UK and Ireland's subsurface has made important contributions to scholarship and prosperity for people and the planet, including economic growth, sustainable use of natural resources, storage of greenhouse gases, and inspiring curiosity about the Earth beneath our feet. This article outlines a vision for an array of seismological instruments spanning the UK and Ireland…
▽ More
Scientific exploration of the UK and Ireland's subsurface has made important contributions to scholarship and prosperity for people and the planet, including economic growth, sustainable use of natural resources, storage of greenhouse gases, and inspiring curiosity about the Earth beneath our feet. This article outlines a vision for an array of seismological instruments spanning the UK and Ireland, UKI Array, augmented by other types of geophysical sensors, to maximise the value offered by existing equipment pools. The mission is to research natural phenomena and structure in the deep and shallow Earth, to solve problems concerning hazards and resources, to connect scientists to schools and the broader public, and thus to inspire a new generation to learn about geophysics. The vision was created through a community driven process of engagement and participation. This paper describes the concept and design of the UKI-Array; a companion paper discusses related opportunities and potential applications.
△ Less
Submitted 24 February, 2026;
originally announced February 2026.
-
The Future of Generative AI in Software Engineering: A Vision from Industry and Academia in the European GENIUS Project
Authors:
Robin Gröpler,
Steffen Klepke,
Jack Johns,
Andreas Dreschinski,
Klaus Schmid,
Benedikt Dornauer,
Eray Tüzün,
Joost Noppen,
Mohammad Reza Mousavi,
Yongjian Tang,
Johannes Viehmann,
Selin Şirin Aslangül,
Beum Seuk Lee,
Adam Ziolkowski,
Eric Zie
Abstract:
Generative AI (GenAI) has recently emerged as a groundbreaking force in Software Engineering, capable of generating code, identifying bugs, recommending fixes, and supporting quality assurance. While its use in coding tasks shows considerable promise, applying GenAI across the entire Software Development Life Cycle (SDLC) has not yet been fully explored. Critical uncertainties in areas such as rel…
▽ More
Generative AI (GenAI) has recently emerged as a groundbreaking force in Software Engineering, capable of generating code, identifying bugs, recommending fixes, and supporting quality assurance. While its use in coding tasks shows considerable promise, applying GenAI across the entire Software Development Life Cycle (SDLC) has not yet been fully explored. Critical uncertainties in areas such as reliability, accountability, security, and data privacy demand deeper investigation and coordinated action. The GENIUS project, comprising over 30 European industrial and academic partners, aims to address these challenges by advancing AI integration across all SDLC phases. It focuses on GenAI's potential, the development of innovative tools, and emerging research challenges, actively shaping the future of software engineering. This vision paper presents a shared perspective on the future of GenAI-driven software engineering, grounded in cross-sector dialogue as well as experiences and findings within the GENIUS consortium. The paper explores four central elements: (1) a structured overview of current challenges in GenAI adoption across the SDLC; (2) a forward-looking vision outlining key technological and methodological advances expected over the next five years; (3) anticipated shifts in the roles and required skill sets of software professionals; and (4) the contribution of GENIUS in realising this transformation through practical tools and industrial validation. This paper focuses on aligning technical innovation with business relevance. It aims to inform both research agendas and industrial strategies, providing a foundation for reliable, scalable, and industry-ready GenAI solutions for software engineering teams.
△ Less
Submitted 6 November, 2025; v1 submitted 3 November, 2025;
originally announced November 2025.