-
Application of mesh refinement to relativistic magnetic reconnection
Authors:
Revathi Jambunathan,
Henry Jones,
Lizzette Corrales,
Hannah Klion,
Michael Rowan,
Andrew Myers,
Weiqun Zhang,
Jean-Luc Vay
Abstract:
During relativistic magnetic reconnection, antiparallel magnetic fields undergo a rapid change in topology, releasing a large amount of energy in the form of non-thermal particle acceleration. This work explores the application of mesh refinement to 2D reconnection simulations to efficiently model the ineherent disparity in length-scales. We have systematically investigated the effects of mesh ref…
▽ More
During relativistic magnetic reconnection, antiparallel magnetic fields undergo a rapid change in topology, releasing a large amount of energy in the form of non-thermal particle acceleration. This work explores the application of mesh refinement to 2D reconnection simulations to efficiently model the ineherent disparity in length-scales. We have systematically investigated the effects of mesh refinement and determined necessary modifications to the algorithm required to mitigate non-physical artifacts at the coarse-fine interface. We have used the ultrahigh-order Pseudo-Spectral Analytical Time-Domain (PSATD) Maxwell solver to analyze how its use can mitigate the numerical dispersion that occurs with the finite-difference time-domain (FDTD) (or ``Yee'') method. Absorbing layers are introduced at the coarse-fine interface to eliminate spurious effects that occur with mesh refinement. We also study how damping the electromagnetic fields and current density in the absorbing layer can help prevent the non-physical accumulation of charge and current density at the coarse-fine interface. Using a mesh refinement ratio of 8 for two-dimensional magnetic reconnection simulations, we obtained good agreement with the high resolution baseline simulation, using only 36% of the macroparticles and 71% of the node-hours needed for the baseline. The methods presented here are especially applicable to 3D systems where higher memory savings are expected than in 2D, enabling comprehensive, computationally efficient 3D reconnection studies in the future.
△ Less
Submitted 16 August, 2024;
originally announced August 2024.
-
In-Situ Assessment of Device-Side Compute Work for Dynamic Load Balancing in a GPU-Accelerated PIC Code
Authors:
Michael E. Rowan,
Axel Huebl,
Kevin N. Gott,
Jack Deslippe,
Maxence Thévenet,
Remi Lehe,
Jean-Luc Vay
Abstract:
Maintaining computational load balance is important to the performant behavior of codes which operate under a distributed computing model. This is especially true for GPU architectures, which can suffer from memory oversubscription if improperly load balanced. We present enhancements to traditional load balancing approaches and explicitly target GPU architectures, exploring the resulting performan…
▽ More
Maintaining computational load balance is important to the performant behavior of codes which operate under a distributed computing model. This is especially true for GPU architectures, which can suffer from memory oversubscription if improperly load balanced. We present enhancements to traditional load balancing approaches and explicitly target GPU architectures, exploring the resulting performance. A key component of our enhancements is the introduction of several GPU-amenable strategies for assessing compute work. These strategies are implemented and benchmarked to find the most optimal data collection methodology for in-situ assessment of GPU compute work. For the fully kinetic particle-in-cell code WarpX, which supports MPI+CUDA parallelism, we investigate the performance of the improved dynamic load balancing via a strong scaling-based performance model and show that, for a laser-ion acceleration test problem run with up to 6144 GPUs on Summit, the enhanced dynamic load balancing achieves from 62%--74% (88% when running on 6 GPUs) of the theoretically predicted maximum speedup; for the 96-GPU case, we find that dynamic load balancing improves performance relative to baselines without load balancing (3.8x speedup) and with static load balancing (1.2x speedup). Our results provide important insights into dynamic load balancing and performance assessment, and are particularly relevant in the context of distributed memory applications ran on GPUs.
△ Less
Submitted 22 April, 2021;
originally announced April 2021.
-
Porting WarpX to GPU-accelerated platforms
Authors:
A. Myers,
A. Almgren,
L. D. Amorim,
J. Bell,
L. Fedeli,
L. Ge,
K. Gott,
D. P. Grote,
M. Hogan,
A. Huebl,
R. Jambunathan,
R. Lehe,
C. Ng,
M. Rowan,
O. Shapoval,
M. Thévenet,
J. -L. Vay,
H. Vincenti,
E. Yang,
N. Zaïm,
W. Zhang,
Y. Zhao,
E. Zoni
Abstract:
WarpX is a general purpose electromagnetic particle-in-cell code that was originally designed to run on many-core CPU architectures. We describe the strategy followed to allow WarpX to use the GPU-accelerated nodes on OLCF's Summit supercomputer, a strategy we believe will extend to the upcoming machines Frontier and Aurora. We summarize the challenges encountered, lessons learned, and give curren…
▽ More
WarpX is a general purpose electromagnetic particle-in-cell code that was originally designed to run on many-core CPU architectures. We describe the strategy followed to allow WarpX to use the GPU-accelerated nodes on OLCF's Summit supercomputer, a strategy we believe will extend to the upcoming machines Frontier and Aurora. We summarize the challenges encountered, lessons learned, and give current performance results on a series of relevant benchmark problems.
△ Less
Submitted 2 September, 2021; v1 submitted 28 January, 2021;
originally announced January 2021.
-
Reconnection-driven particle acceleration in relativistic shear flows
Authors:
Lorenzo Sironi,
Michael E. Rowan,
Ramesh Narayan
Abstract:
Particle energization in shear flows is invoked to explain non-thermal emission from the boundaries of relativistic astrophysical jets. Yet, the physics of particle injection, i.e., the mechanism that allows thermal particles to participate in shear-driven acceleration, remains unknown. With particle-in-cell simulations, we study the development of Kelvin-Helmholtz (KH) instabilities seeded by the…
▽ More
Particle energization in shear flows is invoked to explain non-thermal emission from the boundaries of relativistic astrophysical jets. Yet, the physics of particle injection, i.e., the mechanism that allows thermal particles to participate in shear-driven acceleration, remains unknown. With particle-in-cell simulations, we study the development of Kelvin-Helmholtz (KH) instabilities seeded by the velocity shear between a relativistic magnetically-dominated electron-positron jet and a weakly magnetized electron-ion ambient plasma. We show that, in their nonlinear stages, KH vortices generate kinetic-scale reconnection layers, which efficiently energize the jet particles, thus providing a first-principles mechanism for particle injection into shear-driven acceleration. Our work lends support to spine-sheath models of jet emission - with a fast core/spine surrounded by a slower sheath - and can explain the origin of radio-emitting electrons at the boundaries of relativistic jets.
△ Less
Submitted 24 September, 2020;
originally announced September 2020.
-
Applications and Challenges of Machine Learning to Enable Realistic Cellular Simulations
Authors:
Ritvik Vasan,
Meagan P. Rowan,
Christopher T. Lee,
Gregory R. Johnson,
Padmini Rangamani,
Michael Holst
Abstract:
In this perspective, we examine three key aspects of an end-to-end pipeline for realistic cellular simulations: reconstruction and segmentation of cellular structures; generation of cellular structures; and mesh generation, simulation, and data analysis. We highlight some of the relevant prior work in these distinct but overlapping areas, with a particular emphasis on current use of machine learni…
▽ More
In this perspective, we examine three key aspects of an end-to-end pipeline for realistic cellular simulations: reconstruction and segmentation of cellular structures; generation of cellular structures; and mesh generation, simulation, and data analysis. We highlight some of the relevant prior work in these distinct but overlapping areas, with a particular emphasis on current use of machine learning technologies, as well as on future opportunities.
△ Less
Submitted 12 November, 2019;
originally announced November 2019.
-
Electron and Proton Heating in Transrelativistic Guide Field Reconnection
Authors:
Michael E. Rowan,
Lorenzo Sironi,
Ramesh Narayan
Abstract:
The plasma in low-luminosity accretion flows, such as the one around the black hole at the center of M87 or Sgr A* at our Galactic Center, is expected to be collisioness and two-temperature, with protons hotter than electrons. Here, particle heating is expected to be controlled by magnetic reconnection in the transrelativistic regime $σ_{w}\sim 0.1$-$1$, where the magnetization $σ_{w}$ is the rati…
▽ More
The plasma in low-luminosity accretion flows, such as the one around the black hole at the center of M87 or Sgr A* at our Galactic Center, is expected to be collisioness and two-temperature, with protons hotter than electrons. Here, particle heating is expected to be controlled by magnetic reconnection in the transrelativistic regime $σ_{w}\sim 0.1$-$1$, where the magnetization $σ_{w}$ is the ratio of magnetic energy density to plasma enthalpy density. By means of large-scale 2D particle-in-cell simulations, we explore for a fiducial $σ_w=0.1$ how the dissipated magnetic energy gets partitioned between electrons and protons, as a function of $β_{\rm i}$ (the ratio of proton thermal pressure to magnetic pressure) and of the strength of a guide field $B_{\rm g}$ perpendicular to the reversing field $B_0$. At low $β_{\rm i}\;(\lesssim 0.1)$, we find that the fraction of initial magnetic energy per particle converted into electron irreversible heat is nearly independent of $B_{\rm g}/B_0$, whereas protons get heated much less with increasing $B_{\rm g}/B_0$. As a result, for large $B_{\rm g} /B_{0}$, electrons receive the overwhelming majority of irreversible particle heating (${\sim}93\%$ for $B_{\rm g} /B_{0}=6$). This is significantly different than the antiparallel case $B_{\rm g}/B_0=0$, in which electron irreversible heating accounts for only ${\sim}18\%$ of the total particle heating. At $β_{\rm i} \sim 2$, when both species start already relativistically hot (for our fiducial $σ_w=0.1$), electrons and protons each receive ${\sim}50\%$ of the irreversible particle heating, regardless of the guide field strength. Our results provide important insights into the plasma physics of electron and proton heating in hot accretion flows around supermassive black holes.
△ Less
Submitted 9 March, 2019; v1 submitted 16 January, 2019;
originally announced January 2019.
-
Electron and proton heating in trans-relativistic magnetic reconnection
Authors:
Michael E. Rowan,
Lorenzo Sironi,
Ramesh Narayan
Abstract:
Hot collisionless accretion flows, such as the one in Sgr A$^{*}$ at our Galactic center, provide a unique setting for the investigation of magnetic reconnection. Here, protons are non-relativistic while electrons can be ultra-relativistic. By means of two-dimensional particle-in-cell simulations, we investigate electron and proton heating in the outflows of trans-relativistic reconnection (i.e.,…
▽ More
Hot collisionless accretion flows, such as the one in Sgr A$^{*}$ at our Galactic center, provide a unique setting for the investigation of magnetic reconnection. Here, protons are non-relativistic while electrons can be ultra-relativistic. By means of two-dimensional particle-in-cell simulations, we investigate electron and proton heating in the outflows of trans-relativistic reconnection (i.e., $σ_w\sim 0.1-1$, where the magnetization $σ_w$ is the ratio of magnetic energy density to enthalpy density). For both electrons and protons, we find that heating at high $β_{\rm i}$ (here, $β_{\rm i}$ is the ratio of proton thermal pressure to magnetic pressure) is dominated by adiabatic compression ('adiabatic heating'), while at low $β_{\rm i}$ it is accompanied by a genuine increase in entropy ('irreversible heating'). For our fiducial $σ_w=0.1$, the irreversible heating efficiency at $β_{\rm i}\lesssim 1$ is nearly independent of the electron-to-proton temperature ratio $T_{\rm e}/T_{\rm i}$ (which we vary from $0.1$ up to $1$), and it asymptotes to $\sim 2\%$ of the inflowing magnetic energy in the low-$β_{\rm i}$ limit. Protons are heated more efficiently than electrons at low and moderate $β_{\rm i}$ (by a factor of $\sim7$), whereas the electron and proton heating efficiencies become comparable at $β_{\rm i}\sim 2$ if $T_{\rm e}/T_{\rm i}=1$, when both species start already relativistically hot. We find comparable heating efficiencies between the two species also in the limit of relativistic reconnection ($σ_w\gtrsim 1$). Our results have important implications for the two-temperature nature of collisionless accretion flows, and may provide the sub-grid physics needed in general relativistic MHD simulations.
△ Less
Submitted 11 October, 2017; v1 submitted 15 August, 2017;
originally announced August 2017.
-
Measurement of the $4\: S_{1/2} \rightarrow 6 \: S_{1/2}$ transition frequency in atomic potassium via direct frequency comb spectroscopy
Authors:
J. E. Stalnaker,
H. M. G. Ayer,
J. H. Baron,
A. Nuñez,
M. E. Rowan
Abstract:
We present an experimental determination of the $4 \: S_{1/2} \rightarrow 6\: S_{1/2}$ transition frequency in atomic potassium, $^{39}$K, using direct frequency comb spectroscopy. The output of a stabilized optical frequency comb was used to excite a thermal atomic vapor. The repetition rate of the frequency comb was scanned and the transitions were excited using step-wise two-photon excitation.…
▽ More
We present an experimental determination of the $4 \: S_{1/2} \rightarrow 6\: S_{1/2}$ transition frequency in atomic potassium, $^{39}$K, using direct frequency comb spectroscopy. The output of a stabilized optical frequency comb was used to excite a thermal atomic vapor. The repetition rate of the frequency comb was scanned and the transitions were excited using step-wise two-photon excitation. The center of gravity frequency for the transition was found to be $ν_\textrm{cog} = 822\, 951\, 698.09(13)$ MHz and the measured hyperfine $A$ coefficient of the $6\: S_{1/2}$ state was $21.93(11)$ MHz. The measurements are in agreement with previous values and represent an improvement by a factor of 700 in the uncertainty of the center of gravity measurement.
△ Less
Submitted 9 June, 2017;
originally announced June 2017.
-
Velocity-selective direct frequency-comb spectroscopy of atomic vapors
Authors:
J. E. Stalnaker,
S. L. Chen,
M. E. Rowan,
K. Nguyen,
T. Pradhananga,
C. A. Palm,
D. F. Jackson Kimball
Abstract:
We present an experimental and theoretical investigation of two-photon direct frequency-comb spectroscopy performed through velocity-selective excitation. In particular, we explore the effect of repetition rate on the $\textrm{5S}_{1/2}\rightarrow \textrm{5D}_{3/2, 5/2}$ two-photon transitions excited in a rubidium atomic vapor cell. The transitions occur via step-wise excitation through the…
▽ More
We present an experimental and theoretical investigation of two-photon direct frequency-comb spectroscopy performed through velocity-selective excitation. In particular, we explore the effect of repetition rate on the $\textrm{5S}_{1/2}\rightarrow \textrm{5D}_{3/2, 5/2}$ two-photon transitions excited in a rubidium atomic vapor cell. The transitions occur via step-wise excitation through the $\textrm{5P}_{1/2, 3/2}$ states by use of the direct output of an optical frequency comb. Experiments were performed with two different frequency combs, one with a repetition rate of $\approx 925$ MHz and one with a repetition rate of $\approx 250$ MHz. The experimental spectra are compared to each other and to a theoretical model.
△ Less
Submitted 5 June, 2012;
originally announced June 2012.