-
Madgraph on GPUs and vector CPUs: towards production (The 5-year journey to the first LO release CUDACPP v1.00.00)
Authors:
Andrea Valassi,
Taylor Childers,
Stephan Hageböck,
Daniele Massaro,
Olivier Mattelaer,
Nathan Nichols,
Filip Optolowicz,
Stefan Roiser,
Jørgen Teig,
Zenny Wettersten
Abstract:
The effort to speed up the Madgraph5_aMC@NLO generator by exploiting CPU vectorization and GPUs, which started at the beginning of 2020, has delivered the first production release of the code for leading-order (LO) processes in October 2024. To achieve this goal, many new features, tests and fixes have been implemented in recent months. This process benefitted also from the early feedback of the C…
▽ More
The effort to speed up the Madgraph5_aMC@NLO generator by exploiting CPU vectorization and GPUs, which started at the beginning of 2020, has delivered the first production release of the code for leading-order (LO) processes in October 2024. To achieve this goal, many new features, tests and fixes have been implemented in recent months. This process benefitted also from the early feedback of the CMS experiment. In this contribution, we report on these activities and on the status of the LO software at the time of CHEP2024.
△ Less
Submitted 27 March, 2025;
originally announced March 2025.
-
Madgraph5_aMC@NLO on GPUs and vector CPUs Experience with the first alpha release
Authors:
Stephan Hageboeck,
Taylor Childers,
Walter Hopkins,
Olivier Mattelaer,
Nathan Nichols,
Stefan Roiser,
Jørgen Teig,
Andrea Valassi,
Carl Vuosalo,
Zenny Wettersten
Abstract:
Madgraph5_aMC@NLO is one of the most-frequently used Monte-Carlo event generators at the LHC, and an important consumer of compute resources. The software has been reengineered to maintain the overall look and feel of the user interface while speeding up event generation on CPUs and GPUs. The most computationally intensive part, the calculation of "matrix elements", is offloaded to new implementat…
▽ More
Madgraph5_aMC@NLO is one of the most-frequently used Monte-Carlo event generators at the LHC, and an important consumer of compute resources. The software has been reengineered to maintain the overall look and feel of the user interface while speeding up event generation on CPUs and GPUs. The most computationally intensive part, the calculation of "matrix elements", is offloaded to new implementations optimised for GPUs and for CPU vector instructions, using event-level data parallelism. We present the work to support accelerated leading-order QCD processes, and discuss how this work is going to be released to Madgraph5_aMC@NLO's users.
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
A Portable Parton-Level Event Generator for the High-Luminosity LHC
Authors:
Enrico Bothmann,
Taylor Childers,
Walter Giele,
Stefan Höche,
Joshua Isaacson,
Max Knobbe
Abstract:
The rapid deployment of computing hardware different from the traditional CPU+RAM model in data centers around the world mandates a change in the design of event generators for the Large Hadron Collider, in order to provide economically and ecologically sustainable simulations for the high-luminosity era of the LHC. Parton-level event generation is one of the most computationally demanding parts o…
▽ More
The rapid deployment of computing hardware different from the traditional CPU+RAM model in data centers around the world mandates a change in the design of event generators for the Large Hadron Collider, in order to provide economically and ecologically sustainable simulations for the high-luminosity era of the LHC. Parton-level event generation is one of the most computationally demanding parts of the simulation and is therefore a prime target for improvements. We present a production-ready leading-order parton-level event generation framework capable of utilizing most modern hardware and discuss its performance in the standard candle processes of vector boson and top-quark pair production with up to five additional jets.
△ Less
Submitted 9 August, 2024; v1 submitted 10 November, 2023;
originally announced November 2023.
-
Speeding up Madgraph5 aMC@NLO through CPU vectorization and GPU offloading: towards a first alpha release
Authors:
Andrea Valassi,
Taylor Childers,
Laurence Field,
Stephan Hageböck,
Walter Hopkins,
Olivier Mattelaer,
Nathan Nichols,
Stefan Roiser,
David Smith,
Jorgen Teig,
Carl Vuosalo,
Zenny Wettersten
Abstract:
The matrix element (ME) calculation in any Monte Carlo physics event generator is an ideal fit for implementing data parallelism with lockstep processing on GPUs and vector CPUs. For complex physics processes where the ME calculation is the computational bottleneck of event generation workflows, this can lead to large overall speedups by efficiently exploiting these hardware architectures, which a…
▽ More
The matrix element (ME) calculation in any Monte Carlo physics event generator is an ideal fit for implementing data parallelism with lockstep processing on GPUs and vector CPUs. For complex physics processes where the ME calculation is the computational bottleneck of event generation workflows, this can lead to large overall speedups by efficiently exploiting these hardware architectures, which are now largely underutilized in HEP. In this paper, we present the status of our work on the reengineering of the Madgraph5_aMC@NLO event generator at the time of the ACAT2022 conference. The progress achieved since our previous publication in the ICHEP2022 proceedings is discussed, for our implementations of the ME calculations in vectorized C++, in CUDA and in the SYCL framework, as well as in their integration into the existing MadEvent framework. The outlook towards a first alpha release of the software supporting QCD LO processes usable by the LHC experiments is also discussed.
△ Less
Submitted 9 December, 2023; v1 submitted 31 March, 2023;
originally announced March 2023.
-
Developments in Performance and Portability for MadGraph5_aMC@NLO
Authors:
Andrea Valassi,
Taylor Childers,
Laurence Field,
Stefan Hageböck,
Walter Hopkins,
Olivier Mattelaer,
Nathan Nichols,
Stefan Roiser,
David Smith
Abstract:
Event generators simulate particle interactions using Monte Carlo techniques, providing the primary connection between experiment and theory in experimental high energy physics. These software packages, which are the first step in the simulation worflow of collider experiments, represent approximately 5 to 20% of the annual WLCG usage for the ATLAS and CMS experiments. With computing architectures…
▽ More
Event generators simulate particle interactions using Monte Carlo techniques, providing the primary connection between experiment and theory in experimental high energy physics. These software packages, which are the first step in the simulation worflow of collider experiments, represent approximately 5 to 20% of the annual WLCG usage for the ATLAS and CMS experiments. With computing architectures becoming more heterogeneous, it is important to ensure that these key software frameworks can be run on future systems, large and small. In this contribution, recent progress on porting and speeding up the Madgraph5_aMC@NLO event generator on hybrid architectures, i.e. CPU with GPU accelerators, is discussed. The main focus of this work has been in the calculation of scattering amplitudes and "matrix elements", which is the computational bottleneck of an event generation application. For physics processes limited to QCD leading order, the code generation toolkit has been expanded to produce matrix element calculations using C++ vector instructions on CPUs and using CUDA for NVidia GPUs, as well as using Alpaka, Kokkos and SYCL for multiple CPU and GPU architectures. Performance is reported in terms of matrix element calculations per time on NVidia, Intel, and AMD devices. The status and outlook for the integration of this work into a production release usable by the LHC experiments, with the same functionalities and very similar user interfaces as the current Fortran version, is also described.
△ Less
Submitted 20 October, 2022;
originally announced October 2022.
-
Portability: A Necessary Approach for Future Scientific Software
Authors:
Meghna Bhattacharya,
Paolo Calafiura,
Taylor Childers,
Mark Dewing,
Zhihua Dong,
Oliver Gutsche,
Salman Habib,
Xiangyang Ju,
Michael Kirby,
Kyle Knoepfel,
Matti Kortelainen,
Martin Kwok,
Charles Leggett,
Meifeng Lin,
Vincent R. Pascuzzi,
Alexei Strelchenko,
Brett Viren,
Beomki Yeo,
Haiwang Yu
Abstract:
Today's world of scientific software for High Energy Physics (HEP) is powered by x86 code, while the future will be much more reliant on accelerators like GPUs and FPGAs. The portable parallelization strategies (PPS) project of the High Energy Physics Center for Computational Excellence (HEP/CCE) is investigating solutions for portability techniques that will allow the coding of an algorithm once,…
▽ More
Today's world of scientific software for High Energy Physics (HEP) is powered by x86 code, while the future will be much more reliant on accelerators like GPUs and FPGAs. The portable parallelization strategies (PPS) project of the High Energy Physics Center for Computational Excellence (HEP/CCE) is investigating solutions for portability techniques that will allow the coding of an algorithm once, and the ability to execute it on a variety of hardware products from many vendors, especially including accelerators. We think without these solutions, the scientific success of our experiments and endeavors is in danger, as software development could be expert driven and costly to be able to run on available hardware infrastructure. We think the best solution for the community would be an extension to the C++ standard with a very low entry bar for users, supporting all hardware forms and vendors. We are very far from that ideal though. We argue that in the future, as a community, we need to request and work on portability solutions and strive to reach this ideal.
△ Less
Submitted 15 March, 2022;
originally announced March 2022.
-
HL-LHC Computing Review Stage-2, Common Software Projects: Event Generators
Authors:
The HSF Physics Event Generator WG,
:,
Efe Yazgan,
Josh McFayden,
Andrea Valassi,
Simone Amoroso,
Enrico Bothmann,
Andy Buckley,
John Campbell,
Gurpreet Singh Chahal,
Taylor Childers,
Gloria Corti,
Rikkert Frederix,
Stefano Frixione,
Francesco Giuli,
Alexander Grohsjean,
Stefan Hoeche,
Phil Ilten,
Frank Krauss,
Michal Kreps,
David Lange,
Leif Lonnblad,
Zach Marshall,
Olivier Mattelaer,
Stephen Mrenna
, et al. (14 additional authors not shown)
Abstract:
This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group (WG), as an input to the second phase of the LHCC review of High-Luminosity LHC (HL-LHC) computing, which is due to take place in November 2021. It complements previous documents prepared by the WG in the context of the first phase of the LHCC review in 2020, including in particular the WG paper…
▽ More
This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group (WG), as an input to the second phase of the LHCC review of High-Luminosity LHC (HL-LHC) computing, which is due to take place in November 2021. It complements previous documents prepared by the WG in the context of the first phase of the LHCC review in 2020, including in particular the WG paper on the specific challenges in Monte Carlo event generator software for HL-LHC, which has since been updated and published, and which we are also submitting to the November 2021 review as an integral part of our contribution.
△ Less
Submitted 30 September, 2021;
originally announced September 2021.
-
Porting CMS Heterogeneous Pixel Reconstruction to Kokkos
Authors:
Taylor Childers,
Matti J. Kortelainen,
Martin Kwok,
Alexei Strelchenko,
Yunsong Wang
Abstract:
Programming for a diverse set of compute accelerators in addition to the CPU is a challenge. Maintaining separate source code for each architecture would require lots of effort, and development of new algorithms would be daunting if it had to be repeated many times. Fortunately there are several portability technologies on the market such as Alpaka, Kokkos, and SYCL. These technologies aim to impr…
▽ More
Programming for a diverse set of compute accelerators in addition to the CPU is a challenge. Maintaining separate source code for each architecture would require lots of effort, and development of new algorithms would be daunting if it had to be repeated many times. Fortunately there are several portability technologies on the market such as Alpaka, Kokkos, and SYCL. These technologies aim to improve the developer productivity by making it possible to use the same source code for many different architectures. In this paper we use heterogeneous pixel reconstruction code from the CMS experiment at the CERNL LHC as a realistic use case of a GPU-targeting HEP reconstruction software, and report experience from prototyping a portable version of it using Kokkos. The development was done in a standalone program that attempts to model many of the complexities of a HEP data processing framework such as CMSSW. We also compare the achieved event processing throughput to the original CUDA code and a CPU version of it.
△ Less
Submitted 13 April, 2021;
originally announced April 2021.
-
HL-LHC Computing Review: Common Tools and Community Software
Authors:
HEP Software Foundation,
:,
Thea Aarrestad,
Simone Amoroso,
Markus Julian Atkinson,
Joshua Bendavid,
Tommaso Boccali,
Andrea Bocci,
Andy Buckley,
Matteo Cacciari,
Paolo Calafiura,
Philippe Canal,
Federico Carminati,
Taylor Childers,
Vitaliano Ciulli,
Gloria Corti,
Davide Costanzo,
Justin Gage Dezoort,
Caterina Doglioni,
Javier Mauricio Duarte,
Agnieszka Dziurda,
Peter Elmer,
Markus Elsing,
V. Daniel Elvira,
Giulio Eulisse
, et al. (85 additional authors not shown)
Abstract:
Common and community software packages, such as ROOT, Geant4 and event generators have been a key part of the LHC's success so far and continued development and optimisation will be critical in the future. The challenges are driven by an ambitious physics programme, notably the LHC accelerator upgrade to high-luminosity, HL-LHC, and the corresponding detector upgrades of ATLAS and CMS. In this doc…
▽ More
Common and community software packages, such as ROOT, Geant4 and event generators have been a key part of the LHC's success so far and continued development and optimisation will be critical in the future. The challenges are driven by an ambitious physics programme, notably the LHC accelerator upgrade to high-luminosity, HL-LHC, and the corresponding detector upgrades of ATLAS and CMS. In this document we address the issues for software that is used in multiple experiments (usually even more widely than ATLAS and CMS) and maintained by teams of developers who are either not linked to a particular experiment or who contribute to common software within the context of their experiment activity. We also give space to general considerations for future software and projects that tackle upcoming challenges, no matter who writes it, which is an area where community convergence on best practice is extremely useful.
△ Less
Submitted 31 August, 2020;
originally announced August 2020.
-
Challenges in Monte Carlo event generator software for High-Luminosity LHC
Authors:
The HSF Physics Event Generator WG,
:,
Andrea Valassi,
Efe Yazgan,
Josh McFayden,
Simone Amoroso,
Joshua Bendavid,
Andy Buckley,
Matteo Cacciari,
Taylor Childers,
Vitaliano Ciulli,
Rikkert Frederix,
Stefano Frixione,
Francesco Giuli,
Alexander Grohsjean,
Christian Gütschow,
Stefan Höche,
Walter Hopkins,
Philip Ilten,
Dmitri Konstantinov,
Frank Krauss,
Qiang Li,
Leif Lönnblad,
Fabio Maltoni,
Michelangelo Mangano
, et al. (16 additional authors not shown)
Abstract:
We review the main software and computing challenges for the Monte Carlo physics event generators used by the LHC experiments, in view of the High-Luminosity LHC (HL-LHC) physics programme. This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group as an input to the LHCC review of HL-LHC computing, which has started in May 2020.
We review the main software and computing challenges for the Monte Carlo physics event generators used by the LHC experiments, in view of the High-Luminosity LHC (HL-LHC) physics programme. This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group as an input to the LHCC review of HL-LHC computing, which has started in May 2020.
△ Less
Submitted 18 February, 2021; v1 submitted 28 April, 2020;
originally announced April 2020.
-
Machine Learning in High Energy Physics Community White Paper
Authors:
Kim Albertsson,
Piero Altoe,
Dustin Anderson,
John Anderson,
Michael Andrews,
Juan Pedro Araque Espinosa,
Adam Aurisano,
Laurent Basara,
Adrian Bevan,
Wahid Bhimji,
Daniele Bonacorsi,
Bjorn Burkle,
Paolo Calafiura,
Mario Campanelli,
Louis Capps,
Federico Carminati,
Stefano Carrazza,
Yi-fan Chen,
Taylor Childers,
Yann Coadou,
Elias Coniavitis,
Kyle Cranmer,
Claire David,
Douglas Davis,
Andrea De Simone
, et al. (103 additional authors not shown)
Abstract:
Machine learning has been applied to several problems in particle physics research, beginning with applications to high-level physics analysis in the 1990s and 2000s, followed by an explosion of applications in particle and event identification and reconstruction in the 2010s. In this document we discuss promising future research and development areas for machine learning in particle physics. We d…
▽ More
Machine learning has been applied to several problems in particle physics research, beginning with applications to high-level physics analysis in the 1990s and 2000s, followed by an explosion of applications in particle and event identification and reconstruction in the 2010s. In this document we discuss promising future research and development areas for machine learning in particle physics. We detail a roadmap for their implementation, software and hardware resource requirements, collaborative initiatives with the data science community, academia and industry, and training the particle physics community in data science. The main objective of the document is to connect and motivate these areas of research and development with the physics drivers of the High-Luminosity Large Hadron Collider and future neutrino experiments and identify the resource needs for their implementation. Additionally we identify areas where collaboration with external communities will be of great benefit.
△ Less
Submitted 16 May, 2019; v1 submitted 8 July, 2018;
originally announced July 2018.
-
A Roadmap for HEP Software and Computing R&D for the 2020s
Authors:
Johannes Albrecht,
Antonio Augusto Alves Jr,
Guilherme Amadio,
Giuseppe Andronico,
Nguyen Anh-Ky,
Laurent Aphecetche,
John Apostolakis,
Makoto Asai,
Luca Atzori,
Marian Babik,
Giuseppe Bagliesi,
Marilena Bandieramonte,
Sunanda Banerjee,
Martin Barisits,
Lothar A. T. Bauerdick,
Stefano Belforte,
Douglas Benjamin,
Catrin Bernius,
Wahid Bhimji,
Riccardo Maria Bianchi,
Ian Bird,
Catherine Biscarat,
Jakob Blomer,
Kenneth Bloom,
Tommaso Boccali
, et al. (285 additional authors not shown)
Abstract:
Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for…
▽ More
Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for the HL-LHC in particular, it is critical that all of the collaborating stakeholders agree on the software goals and priorities, and that the efforts complement each other. In this spirit, this white paper describes the R&D activities required to prepare for this software upgrade.
△ Less
Submitted 19 December, 2018; v1 submitted 18 December, 2017;
originally announced December 2017.
-
Adapting the serial Alpgen event generator to simulate LHC collisions on millions of parallel threads
Authors:
J. T. Childers,
T. D. Uram,
T. J. LeCompte,
M. E. Papka,
D. P. Benjamin
Abstract:
As the LHC moves to higher energies and luminosity, the demand for computing resources increases accordingly and will soon outpace the growth of the Worldwide LHC Computing Grid. To meet this greater demand, event generation Monte Carlo was targeted for adaptation to run on Mira, the supercomputer at the Argonne Leadership Computing Facility. Alpgen is a Monte Carlo event generation application th…
▽ More
As the LHC moves to higher energies and luminosity, the demand for computing resources increases accordingly and will soon outpace the growth of the Worldwide LHC Computing Grid. To meet this greater demand, event generation Monte Carlo was targeted for adaptation to run on Mira, the supercomputer at the Argonne Leadership Computing Facility. Alpgen is a Monte Carlo event generation application that is used by LHC experiments in the simulation of collisions that take place in the Large Hadron Collider. This paper details the process by which Alpgen was adapted from a single-processor serial-application to a large-scale parallel-application and the performance that was achieved.
△ Less
Submitted 23 November, 2015;
originally announced November 2015.