Showing 1–50 of 64 results for author: Eriksson, D

MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment

Authors: Hanxian Huang, Igor Fedorov, Andrey Gromov, Bernard Beckerman, Naveen Suda, David Eriksson, Maximilian Balandat, Rylan Conway, Patrick Huber, Chinnadhurai Sankar, Ayushi Dalmia, Zechun Liu, Lemeng Wu, Tarek Elgamal, Adithya Sagar, Vikas Chandra, Raghuraman Krishnamoorthi

Abstract: Real-time AI experiences call for on-device large language models (OD-LLMs) optimized for efficient deployment on resource-constrained hardware. The most useful OD-LLMs produce near-real-time responses and exhibit broad hardware compatibility, maximizing user reach. We present a methodology for designing such models using hardware-in-the-loop architecture search under mobile latency constraints. T… ▽ More Real-time AI experiences call for on-device large language models (OD-LLMs) optimized for efficient deployment on resource-constrained hardware. The most useful OD-LLMs produce near-real-time responses and exhibit broad hardware compatibility, maximizing user reach. We present a methodology for designing such models using hardware-in-the-loop architecture search under mobile latency constraints. This system is amenable to industry-scale deployment: it generates models deployable without custom kernels and compatible with standard mobile runtimes like Executorch. Our methodology avoids specialized attention mechanisms and instead uses attention skipping for long-context acceleration. Our approach jointly optimizes model architecture (layers, dimensions) and attention pattern. To efficiently evaluate candidates, we treat each as a pruned version of a pretrained backbone with inherited weights, thereby achieving high accuracy with minimal continued pretraining. We leverage the low cost of latency evaluation in a staged process: learning an accurate latency model first, then searching for the Pareto-frontier across latency and quality. This yields MobileLLM-Flash, a family of foundation models (350M, 650M, 1.4B) for efficient on-device use with strong capabilities, supporting up to 8k context length. MobileLLM-Flash delivers up to 1.8x and 1.6x faster prefill and decode on mobile CPUs with comparable or superior quality. Our analysis of Pareto-frontier design choices offers actionable principles for OD-LLM design. △ Less

Submitted 27 April, 2026; v1 submitted 16 March, 2026; originally announced March 2026.

Comments: Accepted to ACL Industry Track 2026

Empirical Gaussian Processes

Authors: Jihao Andreas Lin, Sebastian Ament, Louis C. Tiao, David Eriksson, Maximilian Balandat, Eytan Bakshy

Abstract: Gaussian processes (GPs) are powerful and widely used probabilistic regression models, but their effectiveness in practice is often limited by the choice of kernel function. This kernel function is typically handcrafted from a small set of standard functions, a process that requires expert knowledge, results in limited adaptivity to data, and imposes strong assumptions on the hypothesis space. We… ▽ More Gaussian processes (GPs) are powerful and widely used probabilistic regression models, but their effectiveness in practice is often limited by the choice of kernel function. This kernel function is typically handcrafted from a small set of standard functions, a process that requires expert knowledge, results in limited adaptivity to data, and imposes strong assumptions on the hypothesis space. We study Empirical GPs, a principled framework for constructing flexible, data-driven GP priors that overcome these limitations. Rather than relying on standard parametric kernels, we estimate the mean and covariance functions empirically from a corpus of historical observations, enabling the prior to reflect rich, non-trivial covariance structures present in the data. Theoretically, we show that the resulting model converges to the GP that is closest (in KL-divergence sense) to the real data generating process. Practically, we formulate the problem of learning the GP prior from independent datasets as likelihood estimation and derive an Expectation-Maximization algorithm with closed-form updates, allowing the model handle heterogeneous observation locations across datasets. We demonstrate that Empirical GPs achieve competitive performance on learning curve extrapolation and time series forecasting benchmarks. △ Less

Submitted 12 February, 2026; originally announced February 2026.

BONSAI: Bayesian Optimization with Natural Simplicity and Interpretability

Authors: Samuel Daulton, David Eriksson, Maximilian Balandat, Eytan Bakshy

Abstract: Bayesian optimization (BO) is a popular technique for sample-efficient optimization of black-box functions. In many applications, the parameters being tuned come with a carefully engineered default configuration, and practitioners only want to deviate from this default when necessary. Standard BO, however, does not aim to minimize deviation from the default and, in practice, often pushes weakly re… ▽ More Bayesian optimization (BO) is a popular technique for sample-efficient optimization of black-box functions. In many applications, the parameters being tuned come with a carefully engineered default configuration, and practitioners only want to deviate from this default when necessary. Standard BO, however, does not aim to minimize deviation from the default and, in practice, often pushes weakly relevant parameters to the boundary of the search space. This makes it difficult to distinguish between important and spurious changes and increases the burden of vetting recommendations when the optimization objective omits relevant operational considerations. We introduce BONSAI, a default-aware BO policy that prunes low-impact deviations from a default configuration while explicitly controlling the loss in acquisition value. BONSAI is compatible with a variety of acquisition functions, including expected improvement and upper confidence bound (GP-UCB). We theoretically bound the regret incurred by BONSAI, showing that, under certain conditions, it enjoys the same no-regret property as vanilla GP-UCB. Moreover, assuming known ARD lengthscales -- the same assumption underlying GP-UCB regret bounds -- BONSAI provably recovers the relevant-coordinate set at zero acquisition cost, yielding a method that matches the GP-UCB regret rate while recovering the minimal-$\ell_0$ solution -- a guarantee not provided by prior sparse-BO methods. Across many real-world applications, we empirically find that BONSAI substantially reduces the number of non-default parameters in recommended configurations while maintaining competitive optimization performance, with little effect on wall time -- averaging only $1.5\times$ the candidate-generation cost of standard BO, compared to $7$-$34\times$ on average for prior sparse-BO methods (IR, ER, and SEBO). △ Less

Submitted 8 May, 2026; v1 submitted 6 February, 2026; originally announced February 2026.

Comments: 32 pages

Informed Initialization for Bayesian Optimization and Active Learning

Authors: Carl Hvarfner, David Eriksson, Eytan Bakshy, Max Balandat

Abstract: Bayesian Optimization is a widely used method for optimizing expensive black-box functions, relying on probabilistic surrogate models such as Gaussian Processes. The quality of the surrogate model is crucial for good optimization performance, especially in the few-shot setting where only a small number of batches of points can be evaluated. In this setting, the initialization plays a critical role… ▽ More Bayesian Optimization is a widely used method for optimizing expensive black-box functions, relying on probabilistic surrogate models such as Gaussian Processes. The quality of the surrogate model is crucial for good optimization performance, especially in the few-shot setting where only a small number of batches of points can be evaluated. In this setting, the initialization plays a critical role in shaping the surrogate's predictive quality and guiding subsequent optimization. Despite this, practitioners typically rely on (quasi-)random designs to cover the input space. However, such approaches neglect two key factors: (a) space-filling designs may not be desirable to reduce predictive uncertainty, and (b) efficient hyperparameter learning during initialization is essential for high-quality prediction, which may conflict with space-filling designs. To address these limitations, we propose Hyperparameter-Informed Predictive Exploration (HIPE), a novel acquisition strategy that balances predictive uncertainty reduction with hyperparameter learning using information-theoretic principles. We derive a closed-form expression for HIPE in the Gaussian Process setting and demonstrate its effectiveness through extensive experiments in active learning and few-shot BO. Our results show that HIPE outperforms standard initialization strategies in terms of predictive accuracy, hyperparameter identification, and subsequent optimization performance, particularly in large-batch, few-shot settings relevant to many real-world Bayesian Optimization applications. △ Less

Submitted 13 December, 2025; v1 submitted 27 October, 2025; originally announced October 2025.

Comments: 28 pages

Journal ref: NeurIPS 2025

The Deligne-Riemann-Roch isomorphism

Authors: Dennis Eriksson, Gerard Freixas i Montplet

Abstract: This work establishes the geometric component of Deligne's longstanding program on refined Grothendieck-Riemann-Roch formulas expressed through determinants of cohomology. The approach relies on a newly developed universal category of Chern classes together with an associated relative intersection theory. As an example of the applications, we provide a structural description of the coefficients in… ▽ More This work establishes the geometric component of Deligne's longstanding program on refined Grothendieck-Riemann-Roch formulas expressed through determinants of cohomology. The approach relies on a newly developed universal category of Chern classes together with an associated relative intersection theory. As an example of the applications, we provide a structural description of the coefficients in the Knudsen-Mumford expansion and establish a fundamental Mumford-type isomorphism for the alternating product of Griffiths bundles. △ Less

Submitted 2 December, 2025; v1 submitted 5 September, 2025; originally announced September 2025.

Comments: Sequel to "DELIGNE-RIEMANN-ROCH AND INTERSECTION BUNDLES", DOI : 10.5802/jep.254. See Arxiv version arXiv:2305.13129 . Minor updates with respect to presentation

MSC Class: 14C40; 19D23; 14C17; 19D99

Scalable Gaussian Processes with Latent Kronecker Structure

Authors: Jihao Andreas Lin, Sebastian Ament, Maximilian Balandat, David Eriksson, José Miguel Hernández-Lobato, Eytan Bakshy

Abstract: Applying Gaussian processes (GPs) to very large datasets remains a challenge due to limited computational scalability. Matrix structures, such as the Kronecker product, can accelerate operations significantly, but their application commonly entails approximations or unrealistic assumptions. In particular, the most common path to creating a Kronecker-structured kernel matrix is by evaluating a prod… ▽ More Applying Gaussian processes (GPs) to very large datasets remains a challenge due to limited computational scalability. Matrix structures, such as the Kronecker product, can accelerate operations significantly, but their application commonly entails approximations or unrealistic assumptions. In particular, the most common path to creating a Kronecker-structured kernel matrix is by evaluating a product kernel on gridded inputs that can be expressed as a Cartesian product. However, this structure is lost if any observation is missing, breaking the Cartesian product structure, which frequently occurs in real-world data such as time series. To address this limitation, we propose leveraging latent Kronecker structure, by expressing the kernel matrix of observed values as the projection of a latent Kronecker product. In combination with iterative linear system solvers and pathwise conditioning, our method facilitates inference of exact GPs while requiring substantially fewer computational resources than standard iterative methods. We demonstrate that our method outperforms state-of-the-art sparse and variational GPs on real-world datasets with up to five million examples, including robotics, automated machine learning, and climate applications. △ Less

Submitted 16 August, 2025; v1 submitted 7 June, 2025; originally announced June 2025.

Comments: International Conference on Machine Learning 2025

Robust Gaussian Processes via Relevance Pursuit

Authors: Sebastian Ament, Elizabeth Santorella, David Eriksson, Ben Letham, Maximilian Balandat, Eytan Bakshy

Abstract: Gaussian processes (GPs) are non-parametric probabilistic regression models that are popular due to their flexibility, data efficiency, and well-calibrated uncertainty estimates. However, standard GP models assume homoskedastic Gaussian noise, while many real-world applications are subject to non-Gaussian corruptions. Variants of GPs that are more robust to alternative noise models have been propo… ▽ More Gaussian processes (GPs) are non-parametric probabilistic regression models that are popular due to their flexibility, data efficiency, and well-calibrated uncertainty estimates. However, standard GP models assume homoskedastic Gaussian noise, while many real-world applications are subject to non-Gaussian corruptions. Variants of GPs that are more robust to alternative noise models have been proposed, and entail significant trade-offs between accuracy and robustness, and between computational requirements and theoretical guarantees. In this work, we propose and study a GP model that achieves robustness against sparse outliers by inferring data-point-specific noise levels with a sequential selection procedure maximizing the log marginal likelihood that we refer to as relevance pursuit. We show, surprisingly, that the model can be parameterized such that the associated log marginal likelihood is strongly concave in the data-point-specific noise variances, a property rarely found in either robust regression objectives or GP marginal likelihoods. This in turn implies the weak submodularity of the corresponding subset selection problem, and thereby proves approximation guarantees for the proposed algorithm. We compare the model's performance relative to other approaches on diverse regression and Bayesian optimization tasks, including the challenging but common setting of sparse corruptions of the labels within or close to the function range. △ Less

Submitted 7 January, 2025; v1 submitted 31 October, 2024; originally announced October 2024.

Comments: NeurIPS 2024 Article (https://openreview.net/forum?id=5FATPIlWUJ)

Base change conductors through intersection theory and quotient singularities

Authors: Dennis Eriksson, Lars Halvard Halle, Johannes Nicaise

Abstract: We perform a systematic study of the base change conductor for Jacobians. Through the lens of intersection theory and Deligne's Riemann-Roch theorem, we present novel computational approaches for both the tame and wild parts of the base change conductor. Our key results include a general formula of the tame part, as well as a computation of the wild part in terms of Galois quotients of semistable… ▽ More We perform a systematic study of the base change conductor for Jacobians. Through the lens of intersection theory and Deligne's Riemann-Roch theorem, we present novel computational approaches for both the tame and wild parts of the base change conductor. Our key results include a general formula of the tame part, as well as a computation of the wild part in terms of Galois quotients of semistable models of the curves. We treat in detail the case of potential good reduction when the quotient only has weak wild quotient singularities, relying on recent advances by Obus and Wewers. △ Less

Submitted 3 December, 2024; v1 submitted 20 October, 2024; originally announced October 2024.

Comments: 45 pages. Added section 6.4, and corrected typos

Genus one mirror symmetry for intersection of two cubics in $\mathbb{P}^5$

Authors: Dennis Eriksson, Mykola Pochekai

Abstract: This paper establishes BCOV-type genus one mirror symmetry for the intersections of two cubics in $\mathbb{P}^5$. The proof applies previous constructions of the mirror family by the second author and computations of genus one Gromov-Witten invariants by A. Popa. The approach adapts the strategy used for hypersurfaces, as developed by the first author and collaborators, but addresses the distinct… ▽ More This paper establishes BCOV-type genus one mirror symmetry for the intersections of two cubics in $\mathbb{P}^5$. The proof applies previous constructions of the mirror family by the second author and computations of genus one Gromov-Witten invariants by A. Popa. The approach adapts the strategy used for hypersurfaces, as developed by the first author and collaborators, but addresses the distinct geometry involved. A key feature is a systematic usage of toric techniques and related computer aided calculations to determine seemingly otherwise inaccessible invariants. △ Less

Submitted 10 February, 2025; v1 submitted 11 October, 2024; originally announced October 2024.

Comments: A minor improvement in exposition

MSC Class: Primary 14J32; 14J33; Secondary 14Q15; 32G20

Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces

Authors: Aryan Deshwal, Sait Cakmak, Yuhou Xia, David Eriksson

Abstract: Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box functions. However, in settings with very few function evaluations, a successful application of BO may require transferring information from historical experiments. These related experiments may not have exactly the same tunable parameters (search spaces), motivating the need for BO with transfer learni… ▽ More Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box functions. However, in settings with very few function evaluations, a successful application of BO may require transferring information from historical experiments. These related experiments may not have exactly the same tunable parameters (search spaces), motivating the need for BO with transfer learning for heterogeneous search spaces. In this paper, we propose two methods for this setting. The first approach leverages a Gaussian process (GP) model with a conditional kernel to transfer information between different search spaces. Our second approach treats the missing parameters as hyperparameters of the GP model that can be inferred jointly with the other GP hyperparameters or set to fixed values. We show that these two methods perform well on several benchmark problems. △ Less

Submitted 9 September, 2024; originally announced September 2024.

Approximation-Aware Bayesian Optimization

Authors: Natalie Maus, Kyurae Kim, Geoff Pleiss, David Eriksson, John P. Cunningham, Jacob R. Gardner

Abstract: High-dimensional Bayesian optimization (BO) tasks such as molecular design often require 10,000 function evaluations before obtaining meaningful results. While methods like sparse variational Gaussian processes (SVGPs) reduce computational requirements in these settings, the underlying approximations result in suboptimal data acquisitions that slow the progress of optimization. In this paper we mo… ▽ More High-dimensional Bayesian optimization (BO) tasks such as molecular design often require 10,000 function evaluations before obtaining meaningful results. While methods like sparse variational Gaussian processes (SVGPs) reduce computational requirements in these settings, the underlying approximations result in suboptimal data acquisitions that slow the progress of optimization. In this paper we modify SVGPs to better align with the goals of BO: targeting informed data acquisition rather than global posterior fidelity. Using the framework of utility-calibrated variational inference, we unify GP approximation and data acquisition into a joint optimization problem, thereby ensuring optimal decisions under a limited computational budget. Our approach can be used with any decision-theoretic acquisition function and is compatible with trust region methods like TuRBO. We derive efficient joint objectives for the expected improvement and knowledge gradient acquisition functions in both the standard and batch BO settings. Our approach outperforms standard SVGPs on high-dimensional benchmark tasks in control and molecular design. △ Less

Submitted 9 June, 2025; v1 submitted 6 June, 2024; originally announced June 2024.

Numerical scheme for the solution of the "bad" Boussinesq equation

Authors: Christophe Charlier, Daniel Eriksson, Jonatan Lenells

Abstract: We present a numerical scheme for the solution of the initial-value problem for the ``bad'' Boussinesq equation. The accuracy of the scheme is tested by comparison with exact soliton solutions as well as with recently obtained asymptotic formulas for the solution. We present a numerical scheme for the solution of the initial-value problem for the ``bad'' Boussinesq equation. The accuracy of the scheme is tested by comparison with exact soliton solutions as well as with recently obtained asymptotic formulas for the solution. △ Less

Submitted 4 June, 2024; originally announced June 2024.

Comments: 21 pages, 15 figures

The spectral genus of an isolated hypersurface singularity and a conjecture relating to the Milnor number

Authors: Dennis Eriksson, Gerard Freixas i Montplet

Abstract: In this paper, we introduce the notion of spectral genus $\widetilde{p}_{g}$ of a germ of an isolated hypersurface singularity $(\mathbb{C}^{n+1}, 0) \to (\mathbb{C}, 0)$, defined as a sum of small exponents of monodromy eigenvalues. The number of these is equal to the geometric genus $p_{g}$, and hence $\widetilde{p}_g$ can be considered as a secondary invariant to it. We then explore a secondary… ▽ More In this paper, we introduce the notion of spectral genus $\widetilde{p}_{g}$ of a germ of an isolated hypersurface singularity $(\mathbb{C}^{n+1}, 0) \to (\mathbb{C}, 0)$, defined as a sum of small exponents of monodromy eigenvalues. The number of these is equal to the geometric genus $p_{g}$, and hence $\widetilde{p}_g$ can be considered as a secondary invariant to it. We then explore a secondary version of the Durfee conjecture on $p_{g}$, and we predict an inequality between $\widetilde{p}_{g}$ and the Milnor number $μ$, to the effect that $$\widetilde{p}_g\leq\frac{μ-1}{(n+2)!}.$$ We provide evidence by confirming our conjecture in several cases, including homogeneous singularities and singularities with large Newton polyhedra, and quasi-homogeneous or irreducible curve singularities. We also show that a weaker inequality follows from Durfee's conjecture, and hence holds for quasi-homogeneous singularities and curve singularities. Our conjecture is shown to relate closely to the asymptotic behavior of the holomorphic analytic torsion of the sheaf of holomorphic functions on a degeneration of projective varieties, potentially indicating deeper geometric and analytic connections. △ Less

Submitted 3 June, 2024; v1 submitted 6 May, 2024; originally announced May 2024.

Comments: 29 pages. Added a relation with the conjecture of Durfee, and a more detailed discussion about determinants of Laplacians of curves. Also minor improvements of language and presentation

MSC Class: Primary: 32S25; 32S30. Secondary: 14M25; 32S20; 58J52

Unexpected Improvements to Expected Improvement for Bayesian Optimization

Authors: Sebastian Ament, Samuel Daulton, David Eriksson, Maximilian Balandat, Eytan Bakshy

Abstract: Expected Improvement (EI) is arguably the most popular acquisition function in Bayesian optimization and has found countless successful applications, but its performance is often exceeded by that of more recent methods. Notably, EI and its variants, including for the parallel and multi-objective settings, are challenging to optimize because their acquisition values vanish numerically in many regio… ▽ More Expected Improvement (EI) is arguably the most popular acquisition function in Bayesian optimization and has found countless successful applications, but its performance is often exceeded by that of more recent methods. Notably, EI and its variants, including for the parallel and multi-objective settings, are challenging to optimize because their acquisition values vanish numerically in many regions. This difficulty generally increases as the number of observations, dimensionality of the search space, or the number of constraints grow, resulting in performance that is inconsistent across the literature and most often sub-optimal. Herein, we propose LogEI, a new family of acquisition functions whose members either have identical or approximately equal optima as their canonical counterparts, but are substantially easier to optimize numerically. We demonstrate that numerical pathologies manifest themselves in "classic" analytic EI, Expected Hypervolume Improvement (EHVI), as well as their constrained, noisy, and parallel variants, and propose corresponding reformulations that remedy these pathologies. Our empirical results show that members of the LogEI family of acquisition functions substantially improve on the optimization performance of their canonical counterparts and surprisingly, are on par with or exceed the performance of recent state-of-the-art acquisition functions, highlighting the understated role of numerical optimization in the literature. △ Less

Submitted 7 January, 2025; v1 submitted 31 October, 2023; originally announced October 2023.

Comments: NeurIPS 2023 Spotlight (https://openreview.net/forum?id=QFgYOtOkDB)

Deligne-Riemann-Roch and intersection bundles

Authors: Dennis Eriksson, Gerard Freixas i Montplet

Abstract: This article is part of a series of works by the authors with the goal of completing a far-reaching program propounded by Deligne, aiming to extend the codimension one part of the Grothendieck-Riemann-Roch theorem from isomorphism classes of line bundles to isomorphisms thereof. The paper develops a relative functorial intersection theory with values in line bundles, together with a formalism that… ▽ More This article is part of a series of works by the authors with the goal of completing a far-reaching program propounded by Deligne, aiming to extend the codimension one part of the Grothendieck-Riemann-Roch theorem from isomorphism classes of line bundles to isomorphisms thereof. The paper develops a relative functorial intersection theory with values in line bundles, together with a formalism that generalizes previous constructions by Deligne and Elkik, related to the right-hand side of the theorem. △ Less

Submitted 8 June, 2023; v1 submitted 22 May, 2023; originally announced May 2023.

Comments: 99 pages, minor update correcting typos and grammar

MSC Class: Primary: 14C17; 19D99. Secondary: 14C40; 19D23

Ecologically mapped neuronal identity: Towards standardizing activity across heterogeneous experiments

Authors: Kevin Luxem, David Eriksson

Abstract: The brain's diversity of neurons enables a rich behavioral repertoire and flexible adaptation to new situations. Assuming that the ecological pressure has optimized this neuronal variety, we propose exploiting naïve behavior to map the neuronal identity. Here we investigate the feasibility of identifying neurons "ecologically" using their activation for natural behavioral and environmental paramet… ▽ More The brain's diversity of neurons enables a rich behavioral repertoire and flexible adaptation to new situations. Assuming that the ecological pressure has optimized this neuronal variety, we propose exploiting naïve behavior to map the neuronal identity. Here we investigate the feasibility of identifying neurons "ecologically" using their activation for natural behavioral and environmental parameters. Such a neuronal ECO-marker might give a finer granularity than possible with genetic or molecular markers, thereby facilitating the comparison of the functional characteristics of individual neurons across animals. In contrast to a potential mapping using artificial stimuli and trained behavior which have an unlimited parameter space, an ecological mapping is experimentally feasible since it is bounded by the ecology. Home-cage environment is an excellent basis for this ECO-mapping covering an extensive behavioral repertoire and since home-cage behavior is similar across laboratories. We review the possibility of adding area-specific environmental enrichment and automatized behavioral tasks to identify neurons in specific brain areas. In this work, we focus on the visual cortex, motor cortex, prefrontal cortex, and hippocampus. Fundamental to achieving this identification is to take advantage of state-of-the-art behavioral tracking, sensory stimulation protocols, and the plethora of creative behavioral solutions for rodents. We find that motor areas might be easiest to address, followed by prefrontal, hippocampal, and visual areas. The possibility of acquiring a near-complete ecological identification with minimal animal handling, minimal constraints on the main experiment, and data compatibility across laboratories might outweigh the necessity of implanting electrodes or imaging devices. △ Less

Submitted 28 April, 2023; originally announced April 2023.

Comments: 16 Pages

Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddings

Authors: Aryan Deshwal, Sebastian Ament, Maximilian Balandat, Eytan Bakshy, Janardhan Rao Doppa, David Eriksson

Abstract: We consider the problem of optimizing expensive black-box functions over high-dimensional combinatorial spaces which arises in many science, engineering, and ML applications. We use Bayesian Optimization (BO) and propose a novel surrogate modeling approach for efficiently handling a large number of binary and categorical parameters. The key idea is to select a number of discrete structures from th… ▽ More We consider the problem of optimizing expensive black-box functions over high-dimensional combinatorial spaces which arises in many science, engineering, and ML applications. We use Bayesian Optimization (BO) and propose a novel surrogate modeling approach for efficiently handling a large number of binary and categorical parameters. The key idea is to select a number of discrete structures from the input space (the dictionary) and use them to define an ordinal embedding for high-dimensional combinatorial structures. This allows us to use existing Gaussian process models for continuous spaces. We develop a principled approach based on binary wavelets to construct dictionaries for binary spaces, and propose a randomized construction method that generalizes to categorical spaces. We provide theoretical justification to support the effectiveness of the dictionary-based embeddings. Our experiments on diverse real-world benchmarks demonstrate the effectiveness of our proposed surrogate modeling approach over state-of-the-art BO methods. △ Less

Submitted 3 March, 2023; originally announced March 2023.

Comments: Appearing in AISTATS 2023

Discovering Many Diverse Solutions with Bayesian Optimization

Authors: Natalie Maus, Kaiwen Wu, David Eriksson, Jacob Gardner

Abstract: Bayesian optimization (BO) is a popular approach for sample-efficient optimization of black-box objective functions. While BO has been successfully applied to a wide range of scientific applications, traditional approaches to single-objective BO only seek to find a single best solution. This can be a significant limitation in situations where solutions may later turn out to be intractable. For exa… ▽ More Bayesian optimization (BO) is a popular approach for sample-efficient optimization of black-box objective functions. While BO has been successfully applied to a wide range of scientific applications, traditional approaches to single-objective BO only seek to find a single best solution. This can be a significant limitation in situations where solutions may later turn out to be intractable. For example, a designed molecule may turn out to violate constraints that can only be reasonably evaluated after the optimization process has concluded. To address this issue, we propose Rank-Ordered Bayesian Optimization with Trust-regions (ROBOT) which aims to find a portfolio of high-performing solutions that are diverse according to a user-specified diversity metric. We evaluate ROBOT on several real-world applications and show that it can discover large sets of high-performing diverse solutions while requiring few additional function evaluations compared to finding a single best solution. △ Less

Submitted 2 May, 2023; v1 submitted 19 October, 2022; originally announced October 2022.

Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization

Authors: Samuel Daulton, Xingchen Wan, David Eriksson, Maximilian Balandat, Michael A. Osborne, Eytan Bakshy

Abstract: Optimizing expensive-to-evaluate black-box functions of discrete (and potentially continuous) design parameters is a ubiquitous problem in scientific and engineering applications. Bayesian optimization (BO) is a popular, sample-efficient method that leverages a probabilistic surrogate model and an acquisition function (AF) to select promising designs to evaluate. However, maximizing the AF over mi… ▽ More Optimizing expensive-to-evaluate black-box functions of discrete (and potentially continuous) design parameters is a ubiquitous problem in scientific and engineering applications. Bayesian optimization (BO) is a popular, sample-efficient method that leverages a probabilistic surrogate model and an acquisition function (AF) to select promising designs to evaluate. However, maximizing the AF over mixed or high-cardinality discrete search spaces is challenging standard gradient-based methods cannot be used directly or evaluating the AF at every point in the search space would be computationally prohibitive. To address this issue, we propose using probabilistic reparameterization (PR). Instead of directly optimizing the AF over the search space containing discrete parameters, we instead maximize the expectation of the AF over a probability distribution defined by continuous parameters. We prove that under suitable reparameterizations, the BO policy that maximizes the probabilistic objective is the same as that which maximizes the AF, and therefore, PR enjoys the same regret bounds as the original BO policy using the underlying AF. Moreover, our approach provably converges to a stationary point of the probabilistic objective under gradient ascent using scalable, unbiased estimators of both the probabilistic objective and its gradient. Therefore, as the number of starting points and gradient steps increase, our approach will recover of a maximizer of the AF (an often-neglected requisite for commonly used BO regret bounds). We validate our approach empirically and demonstrate state-of-the-art optimization performance on a wide range of real-world applications. PR is complementary to (and benefits) recent work and naturally generalizes to settings with multiple objectives and black-box constraints. △ Less

Submitted 18 October, 2022; originally announced October 2022.

Comments: To appear in Advances in Neural Information Processing Systems 35, 2022. Code available at: https://github.com/facebookresearch/bo_pr

Sparse Bayesian Optimization

Authors: Sulin Liu, Qing Feng, David Eriksson, Benjamin Letham, Eytan Bakshy

Abstract: Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box objective functions. However, the application of BO to areas such as recommendation systems often requires taking the interpretability and simplicity of the configurations into consideration, a setting that has not been previously studied in the BO literature. To make BO useful for this setting, we pres… ▽ More Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box objective functions. However, the application of BO to areas such as recommendation systems often requires taking the interpretability and simplicity of the configurations into consideration, a setting that has not been previously studied in the BO literature. To make BO useful for this setting, we present several regularization-based approaches that allow us to discover sparse and more interpretable configurations. We propose a novel differentiable relaxation based on homotopy continuation that makes it possible to target sparsity by working directly with $L_0$ regularization. We identify failure modes for regularized BO and develop a hyperparameter-free method, sparsity exploring Bayesian optimization (SEBO) that seeks to simultaneously maximize a target objective and sparsity. SEBO and methods based on fixed regularization are evaluated on synthetic and real-world problems, and we show that we are able to efficiently optimize for sparsity. △ Less

Submitted 3 March, 2023; v1 submitted 3 March, 2022; originally announced March 2022.

Autonomous Heavy-Duty Mobile Machinery: A Multidisciplinary Collaborative Challenge

Authors: Tyrone Machado, David Fassbender, Abdolreza Taheri, Daniel Eriksson, Himanshu Gupta, Amirmasoud Molaei, Paolo Forte, Prashant Rai, Reza Ghabcheloo, Saku Mäkinen, Achim Lilienthal, Henrik Andreasson, Marcus Geimer

Abstract: Heavy-duty mobile machines (HDMMs) are a wide range of machinery used in diverse and critical application areas which are currently facing several issues like skilled labor shortage, poor safety records, and harsh work environments. Consequently, efforts are underway to increase automation in HDMMs for increased productivity and safety, eventually transitioning to operator-less autonomous HDMMs to… ▽ More Heavy-duty mobile machines (HDMMs) are a wide range of machinery used in diverse and critical application areas which are currently facing several issues like skilled labor shortage, poor safety records, and harsh work environments. Consequently, efforts are underway to increase automation in HDMMs for increased productivity and safety, eventually transitioning to operator-less autonomous HDMMs to address skilled labor shortages. However, HDMM are complex machines requiring continuous physical and cognitive inputs from human-operators. Thus, developing autonomous HDMM is a huge challenge, with current research and developments being performed in several independent research domains. Through this study, we use the bounded rationality concept to propose multidisciplinary collaborations for new autonomous HDMMs and apply the transaction cost economics framework to suggest future implications in the HDMM industry. Furthermore, we introduce a conceptual understanding of collaborations in the autonomous HDMM as a unified approach, while highlighting the practical implications and challenges of the complex nature of such multidisciplinary collaborations. The collaborative challenges and potentials are mapped out between the following topics: mechanical systems, AI methods, software systems, sensors, connectivity, simulations and process optimization, business cases, organization theories, and finally, regulatory frameworks. △ Less

Submitted 9 January, 2022; v1 submitted 5 December, 2021; originally announced December 2021.

Comments: published in 2021 IEEE International Conference on Technology and Entrepreneurship (ICTE)

Journal ref: 2021 IEEE International Conference on Technology and Entrepreneurship (ICTE), 2021, Kaunas, Lithuania, pp. 1-8

Multi-Objective Bayesian Optimization over High-Dimensional Search Spaces

Authors: Samuel Daulton, David Eriksson, Maximilian Balandat, Eytan Bakshy

Abstract: Many real world scientific and industrial applications require optimizing multiple competing black-box objectives. When the objectives are expensive-to-evaluate, multi-objective Bayesian optimization (BO) is a popular approach because of its high sample efficiency. However, even with recent methodological advances, most existing multi-objective BO methods perform poorly on search spaces with more… ▽ More Many real world scientific and industrial applications require optimizing multiple competing black-box objectives. When the objectives are expensive-to-evaluate, multi-objective Bayesian optimization (BO) is a popular approach because of its high sample efficiency. However, even with recent methodological advances, most existing multi-objective BO methods perform poorly on search spaces with more than a few dozen parameters and rely on global surrogate models that scale cubically with the number of observations. In this work we propose MORBO, a scalable method for multi-objective BO over high-dimensional search spaces. MORBO identifies diverse globally optimal solutions by performing BO in multiple local regions of the design space in parallel using a coordinated strategy. We show that MORBO significantly advances the state-of-the-art in sample efficiency for several high-dimensional synthetic problems and real world applications, including an optical display design problem and a vehicle design problem with 146 and 222 parameters, respectively. On these problems, where existing BO algorithms fail to scale and perform well, MORBO provides practitioners with order-of-magnitude improvements in sample efficiency over the current approach. △ Less

Submitted 15 June, 2022; v1 submitted 22 September, 2021; originally announced September 2021.

Comments: To appear at UAI 2022. 24 pages

Complex Chern--Simons bundles in the relative setting

Authors: Dennis Eriksson, Gerard Freixas i Montplet, Richard A. Wentworth

Abstract: Complex Chern-Simons bundles are line bundles with connection, originating in the study of quantization of moduli spaces of flat connections with complex gauge groups. In this paper we introduce and study these bundles in the families setting. The central object is a functorial direct image of characteristic classes of vector bundles with connections, for which we develop a formalism. Our strate… ▽ More Complex Chern-Simons bundles are line bundles with connection, originating in the study of quantization of moduli spaces of flat connections with complex gauge groups. In this paper we introduce and study these bundles in the families setting. The central object is a functorial direct image of characteristic classes of vector bundles with connections, for which we develop a formalism. Our strategy elaborates on Deligne-Elkik's intersection bundles, and a refined Chern-Simons theory which parallels the use of Bott-Chern classes in Arakelov geometry. In the context of moduli spaces, we are confronted with flat relative connections on families of Riemann surfaces. To be able to rely on the functorial approach, we prove canonical extension results to global connections, inspired by the deformation theory of harmonic maps in non-abelian Hodge theory. The relative complex Chern-Simons bundle $\mathcal{L}_{CS}$ is then defined as a functorial direct image of the second Chern class on the relative moduli space of flat vector bundles. We establish the crystalline nature of $\mathcal{L}_{CS}$, and the existence of a holomorphic extension of natural metrics from Arakelov geometry. The curvature of $\mathcal{L}_{CS}$ can be expressed in terms of the Atiyah-Bott-Goldman form, in agreement with the classical topological approach. To highlight a few applications, we first mention a characterization of projective structures of Riemann surfaces in terms of connections on intersection bundles. In particular, we settle a conjecture of Bertola-Korotkin-Norton on the comparison between the Bergman and the Bers projective structures. This is the problem of determining the accessory parameters of quasi-Fuchsian uniformizations. A conjecture of Cappell-Miller is also established, to the effect that their holomorphic torsion satisfies a Riemann-Roch formula. △ Less

Submitted 16 March, 2022; v1 submitted 5 September, 2021; originally announced September 2021.

Comments: 138 pages. The current version improves the exposition and corrects some minor shortcomings. Main statements are unchanged

Latency-Aware Neural Architecture Search with Multi-Objective Bayesian Optimization

Authors: David Eriksson, Pierce I-Jen Chuang, Samuel Daulton, Peng Xia, Akshat Shrivastava, Arun Babu, Shicong Zhao, Ahmed Aly, Ganesh Venkatesh, Maximilian Balandat

Abstract: When tuning the architecture and hyperparameters of large machine learning models for on-device deployment, it is desirable to understand the optimal trade-offs between on-device latency and model accuracy. In this work, we leverage recent methodological advances in Bayesian optimization over high-dimensional search spaces and multi-objective Bayesian optimization to efficiently explore these trad… ▽ More When tuning the architecture and hyperparameters of large machine learning models for on-device deployment, it is desirable to understand the optimal trade-offs between on-device latency and model accuracy. In this work, we leverage recent methodological advances in Bayesian optimization over high-dimensional search spaces and multi-objective Bayesian optimization to efficiently explore these trade-offs for a production-scale on-device natural language understanding model at Facebook. △ Less

Submitted 25 June, 2021; v1 submitted 22 June, 2021; originally announced June 2021.

Comments: To Appear at the 8th ICML Workshop on Automated Machine Learning, ICML 2021

A Nonmyopic Approach to Cost-Constrained Bayesian Optimization

Authors: Eric Hans Lee, David Eriksson, Valerio Perrone, Matthias Seeger

Abstract: Bayesian optimization (BO) is a popular method for optimizing expensive-to-evaluate black-box functions. BO budgets are typically given in iterations, which implicitly assumes each evaluation has the same cost. In fact, in many BO applications, evaluation costs vary significantly in different regions of the search space. In hyperparameter optimization, the time spent on neural network training inc… ▽ More Bayesian optimization (BO) is a popular method for optimizing expensive-to-evaluate black-box functions. BO budgets are typically given in iterations, which implicitly assumes each evaluation has the same cost. In fact, in many BO applications, evaluation costs vary significantly in different regions of the search space. In hyperparameter optimization, the time spent on neural network training increases with layer size; in clinical trials, the monetary cost of drug compounds vary; and in optimal control, control actions have differing complexities. Cost-constrained BO measures convergence with alternative cost metrics such as time, money, or energy, for which the sample efficiency of standard BO methods is ill-suited. For cost-constrained BO, cost efficiency is far more important than sample efficiency. In this paper, we formulate cost-constrained BO as a constrained Markov decision process (CMDP), and develop an efficient rollout approximation to the optimal CMDP policy that takes both the cost and future iterations into account. We validate our method on a collection of hyperparameter optimization problems as well as a sensor set selection application. △ Less

Submitted 10 June, 2021; originally announced June 2021.

Comments: To appear in UAI 2021

Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020

Authors: Ryan Turner, David Eriksson, Michael McCourt, Juha Kiili, Eero Laaksonen, Zhen Xu, Isabelle Guyon

Abstract: This paper presents the results and insights from the black-box optimization (BBO) challenge at NeurIPS 2020 which ran from July-October, 2020. The challenge emphasized the importance of evaluating derivative-free optimizers for tuning the hyperparameters of machine learning models. This was the first black-box optimization challenge with a machine learning emphasis. It was based on tuning (valida… ▽ More This paper presents the results and insights from the black-box optimization (BBO) challenge at NeurIPS 2020 which ran from July-October, 2020. The challenge emphasized the importance of evaluating derivative-free optimizers for tuning the hyperparameters of machine learning models. This was the first black-box optimization challenge with a machine learning emphasis. It was based on tuning (validation set) performance of standard machine learning models on real datasets. This competition has widespread impact as black-box optimization (e.g., Bayesian optimization) is relevant for hyperparameter tuning in almost every machine learning project as well as many applications outside of machine learning. The final leaderboard was determined using the optimization performance on held-out (hidden) objective functions, where the optimizers ran without human intervention. Baselines were set using the default settings of several open-source black-box optimization packages as well as random search. △ Less

Submitted 31 August, 2021; v1 submitted 20 April, 2021; originally announced April 2021.

High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspaces

Authors: David Eriksson, Martin Jankowiak

Abstract: Bayesian optimization (BO) is a powerful paradigm for efficient optimization of black-box objective functions. High-dimensional BO presents a particular challenge, in part because the curse of dimensionality makes it difficult to define -- as well as do inference over -- a suitable class of surrogate models. We argue that Gaussian process surrogate models defined on sparse axis-aligned subspaces o… ▽ More Bayesian optimization (BO) is a powerful paradigm for efficient optimization of black-box objective functions. High-dimensional BO presents a particular challenge, in part because the curse of dimensionality makes it difficult to define -- as well as do inference over -- a suitable class of surrogate models. We argue that Gaussian process surrogate models defined on sparse axis-aligned subspaces offer an attractive compromise between flexibility and parsimony. We demonstrate that our approach, which relies on Hamiltonian Monte Carlo for inference, can rapidly identify sparse subspaces relevant to modeling the unknown objective function, enabling sample-efficient high-dimensional BO. In an extensive suite of experiments comparing to existing methods for high-dimensional BO we demonstrate that our algorithm, Sparse Axis-Aligned Subspace BO (SAASBO), achieves excellent performance on several synthetic and real-world problems without the need to set problem-specific hyperparameters. △ Less

Submitted 10 June, 2021; v1 submitted 27 February, 2021; originally announced March 2021.

Comments: To appear in UAI 2021

Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization

Authors: Geoff Pleiss, Martin Jankowiak, David Eriksson, Anil Damle, Jacob R. Gardner

Abstract: Matrix square roots and their inverses arise frequently in machine learning, e.g., when sampling from high-dimensional Gaussians $\mathcal{N}(\mathbf 0, \mathbf K)$ or whitening a vector $\mathbf b$ against covariance matrix $\mathbf K$. While existing methods typically require $O(N^3)$ computation, we introduce a highly-efficient quadratic-time algorithm for computing $\mathbf K^{1/2} \mathbf b$,… ▽ More Matrix square roots and their inverses arise frequently in machine learning, e.g., when sampling from high-dimensional Gaussians $\mathcal{N}(\mathbf 0, \mathbf K)$ or whitening a vector $\mathbf b$ against covariance matrix $\mathbf K$. While existing methods typically require $O(N^3)$ computation, we introduce a highly-efficient quadratic-time algorithm for computing $\mathbf K^{1/2} \mathbf b$, $\mathbf K^{-1/2} \mathbf b$, and their derivatives through matrix-vector multiplication (MVMs). Our method combines Krylov subspace methods with a rational approximation and typically achieves $4$ decimal places of accuracy with fewer than $100$ MVMs. Moreover, the backward pass requires little additional computation. We demonstrate our method's applicability on matrices as large as $50,\!000 \times 50,\!000$ - well beyond traditional methods - with little approximation error. Applying this increased scalability to variational Gaussian processes, Bayesian optimization, and Gibbs sampling results in more powerful models with higher accuracy. △ Less

Submitted 30 November, 2020; v1 submitted 19 June, 2020; originally announced June 2020.

Comments: NeurIPS 2020

Efficient Rollout Strategies for Bayesian Optimization

Authors: Eric Hans Lee, David Eriksson, Bolong Cheng, Michael McCourt, David Bindel

Abstract: Bayesian optimization (BO) is a class of sample-efficient global optimization methods, where a probabilistic model conditioned on previous observations is used to determine future evaluations via the optimization of an acquisition function. Most acquisition functions are myopic, meaning that they only consider the impact of the next function evaluation. Non-myopic acquisition functions consider th… ▽ More Bayesian optimization (BO) is a class of sample-efficient global optimization methods, where a probabilistic model conditioned on previous observations is used to determine future evaluations via the optimization of an acquisition function. Most acquisition functions are myopic, meaning that they only consider the impact of the next function evaluation. Non-myopic acquisition functions consider the impact of the next $h$ function evaluations and are typically computed through rollout, in which $h$ steps of BO are simulated. These rollout acquisition functions are defined as $h$-dimensional integrals, and are expensive to compute and optimize. We show that a combination of quasi-Monte Carlo, common random numbers, and control variates significantly reduce the computational burden of rollout. We then formulate a policy-search based approach that removes the need to optimize the rollout acquisition function. Finally, we discuss the qualitative behavior of rollout policies in the setting of multi-modal objectives and model error. △ Less

Submitted 18 June, 2020; v1 submitted 24 February, 2020; originally announced February 2020.

Comments: To appear in UAI 2020

Scalable Constrained Bayesian Optimization

Authors: David Eriksson, Matthias Poloczek

Abstract: The global optimization of a high-dimensional black-box function under black-box constraints is a pervasive task in machine learning, control, and engineering. These problems are challenging since the feasible set is typically non-convex and hard to find, in addition to the curses of dimensionality and the heterogeneity of the underlying functions. In particular, these characteristics dramatically… ▽ More The global optimization of a high-dimensional black-box function under black-box constraints is a pervasive task in machine learning, control, and engineering. These problems are challenging since the feasible set is typically non-convex and hard to find, in addition to the curses of dimensionality and the heterogeneity of the underlying functions. In particular, these characteristics dramatically impact the performance of Bayesian optimization methods, that otherwise have become the de facto standard for sample-efficient optimization in unconstrained settings, leaving practitioners with evolutionary strategies or heuristics. We propose the scalable constrained Bayesian optimization (SCBO) algorithm that overcomes the above challenges and pushes the applicability of Bayesian optimization far beyond the state-of-the-art. A comprehensive experimental evaluation demonstrates that SCBO achieves excellent results on a variety of benchmarks. To this end, we propose two new control problems that we expect to be of independent value for the scientific community. △ Less

Submitted 28 February, 2021; v1 submitted 19 February, 2020; originally announced February 2020.

Comments: To appear in Proceedings of AISTATS 2021

On genus one mirror symmetry in higher dimensions and the BCOV conjectures

Authors: Gerard Freixas I Montplet, Dennis Eriksson, Christophe Mourougane

Abstract: The mathematical physicists Bershadsky-Cecotti-Ooguri-Vafa (BCOV) proposed, in a seminal article from '94, a conjecture extending genus zero mirror symmetry to higher genera. With a view towards a refined formulation of the Grothendieck-Riemann-Roch theorem, we offer a mathematical description of the BCOV conjecture at genus one. As an application of the arithmetic Riemann-Roch theorem of Gillet-S… ▽ More The mathematical physicists Bershadsky-Cecotti-Ooguri-Vafa (BCOV) proposed, in a seminal article from '94, a conjecture extending genus zero mirror symmetry to higher genera. With a view towards a refined formulation of the Grothendieck-Riemann-Roch theorem, we offer a mathematical description of the BCOV conjecture at genus one. As an application of the arithmetic Riemann-Roch theorem of Gillet-Soulé and of our previous results on the BCOV invariant, we establish this conjecture for Calabi-Yau hypersurfaces in projective spaces. Our contribution takes place on the $B$-side, and together with the work of Zinger on the $A$-side, it provides the first complete examples of the mirror symmetry program in higher dimensions. The case of quintic threefolds was studied by Fang-Lu-Yoshikawa. Our approach also lends itself to arithmetic considerations of the BCOV invariant, and we study a Chowla-Selberg type theorem expressing it in terms of special $Γ$ values for certain Calabi-Yau manifolds with complex multiplication. △ Less

Submitted 24 June, 2022; v1 submitted 15 November, 2019; originally announced November 2019.

Comments: Final version, to appear in Forum of Mathematics, Pi

MSC Class: Primary: 14J32; 14J33; 58J52. Secondary: 32G20

Journal ref: Forum of Mathematics, Pi (2022) 10, E19

Scalable Global Optimization via Local Bayesian Optimization

Authors: David Eriksson, Michael Pearce, Jacob R Gardner, Ryan Turner, Matthias Poloczek

Abstract: Bayesian optimization has recently emerged as a popular method for the sample-efficient optimization of expensive black-box functions. However, the application to high-dimensional problems with several thousand observations remains challenging, and on difficult problems Bayesian optimization is often not competitive with other paradigms. In this paper we take the view that this is due to the impli… ▽ More Bayesian optimization has recently emerged as a popular method for the sample-efficient optimization of expensive black-box functions. However, the application to high-dimensional problems with several thousand observations remains challenging, and on difficult problems Bayesian optimization is often not competitive with other paradigms. In this paper we take the view that this is due to the implicit homogeneity of the global probabilistic models and an overemphasized exploration that results from global acquisition. This motivates the design of a local probabilistic approach for global optimization of large-scale high-dimensional problems. We propose the $\texttt{TuRBO}$ algorithm that fits a collection of local models and performs a principled global allocation of samples across these models via an implicit bandit approach. A comprehensive evaluation demonstrates that $\texttt{TuRBO}$ outperforms state-of-the-art methods from machine learning and operations research on problems spanning reinforcement learning, robotics, and the natural sciences. △ Less

Submitted 24 February, 2020; v1 submitted 3 October, 2019; originally announced October 2019.

Comments: Appears in NeurIPS 2019 as a spotlight paper

Journal ref: In Advances in Neural Information Processing Systems 32, pages 5497-5508. 2019

Nonproper intersection products and generalized cycles

Authors: Mats Andersson, Dennis Eriksson, Håkan Samuelsson Kalm, Elizabeth Wulcan, Alain Yger

Abstract: In this article we develop intersection theory in terms of the $\mathcal{B}$-group of a reduced analytic space. This group was introduced in a previous work as an analogue of the Chow group; it is generated by currents that are direct images of Chern forms and it contains all usual cycles. However, contrary to Chow classes, the $\mathcal{B}$-classes have well-defined multiplicities at each point.… ▽ More In this article we develop intersection theory in terms of the $\mathcal{B}$-group of a reduced analytic space. This group was introduced in a previous work as an analogue of the Chow group; it is generated by currents that are direct images of Chern forms and it contains all usual cycles. However, contrary to Chow classes, the $\mathcal{B}$-classes have well-defined multiplicities at each point. We focus on a $\mathcal{B}$-analogue of the intersection theory based on the Stückrad-Vogel procedure and the join construction in projective space. Our approach provides global $\mathcal{B}$-classes which satisfy a Bézout theorem and have the expected local intersection numbers. An essential feature is that we take averages, over various auxiliary choices, by integration. We also introduce $\mathcal{B}$-analogues of more classical constructions of intersections using the Gysin map of the diagonal. These constructions are connected via a $\mathcal{B}$-variant of van Gastel's formulas. Furthermore, we prove that our intersections coincide with the classical ones on cohomology level. △ Less

Submitted 30 August, 2019; originally announced August 2019.

pySOT and POAP: An event-driven asynchronous framework for surrogate optimization

Authors: David Eriksson, David Bindel, Christine A. Shoemaker

Abstract: This paper describes Plumbing for Optimization with Asynchronous Parallelism (POAP) and the Python Surrogate Optimization Toolbox (pySOT). POAP is an event-driven framework for building and combining asynchronous optimization strategies, designed for global optimization of expensive functions where concurrent function evaluations are useful. POAP consists of three components: a worker pool capable… ▽ More This paper describes Plumbing for Optimization with Asynchronous Parallelism (POAP) and the Python Surrogate Optimization Toolbox (pySOT). POAP is an event-driven framework for building and combining asynchronous optimization strategies, designed for global optimization of expensive functions where concurrent function evaluations are useful. POAP consists of three components: a worker pool capable of function evaluations, strategies to propose evaluations or other actions, and a controller that mediates the interaction between the workers and strategies. pySOT is a collection of synchronous and asynchronous surrogate optimization strategies, implemented in the POAP framework. We support the stochastic RBF method by Regis and Shoemaker along with various extensions of this method, and a general surrogate optimization strategy that covers most Bayesian optimization methods. We have implemented many different surrogate models, experimental designs, acquisition functions, and a large set of test problems. We make an extensive comparison between synchronous and asynchronous parallelism and find that the advantage of asynchronous computation increases as the variance of the evaluation time or number of processors increases. We observe a close to linear speed-up with 4, 8, and 16 processors in both the synchronous and asynchronous setting. △ Less

Submitted 30 July, 2019; originally announced August 2019.

Global representation of Segre numbers by Monge-Ampère products

Authors: Mats Andersson, Dennis Eriksson, Håkan Samuelsson Kalm, Elizabeth Wulcan, Alain Yger

Abstract: On a reduced analytic space $X$ we introduce the concept of a generalized cycle, which extends the notion of a formal sum of analytic subspaces to include also a form part. We then consider a suitable equivalence relation and corresponding quotient $\mathcal{B}(X)$ that we think of as an analogue of the Chow group and a refinement of de Rham cohomology. This group allows us to study both global an… ▽ More On a reduced analytic space $X$ we introduce the concept of a generalized cycle, which extends the notion of a formal sum of analytic subspaces to include also a form part. We then consider a suitable equivalence relation and corresponding quotient $\mathcal{B}(X)$ that we think of as an analogue of the Chow group and a refinement of de Rham cohomology. This group allows us to study both global and local intersection theoretic properties. We provide many $\mathcal{B}$-analogues of classical intersection theoretic constructions: For an analytic subspace $V\subset X$ we define a $\mathcal{B}$-Segre class, which is an element of $\mathcal{B}(X)$ with support in $V$. It satisfies a global King formula and, in particular, its multiplicities at each point coincide with the Segre numbers of $V$. When $V$ is cut out by a section of a vector bundle we interpret this class as a Monge-Ampère-type product. For regular embeddings we construct a $\mathcal{B}$-analogue of the Gysin morphism. △ Less

Submitted 13 March, 2020; v1 submitted 7 December, 2018; originally announced December 2018.

Scaling Gaussian Process Regression with Derivatives

Authors: David Eriksson, Kun Dong, Eric Hans Lee, David Bindel, Andrew Gordon Wilson

Abstract: Gaussian processes (GPs) with derivatives are useful in many applications, including Bayesian optimization, implicit surface reconstruction, and terrain reconstruction. Fitting a GP to function values and derivatives at $n$ points in $d$ dimensions requires linear solves and log determinants with an ${n(d+1) \times n(d+1)}$ positive definite matrix -- leading to prohibitive $\mathcal{O}(n^3d^3)$ c… ▽ More Gaussian processes (GPs) with derivatives are useful in many applications, including Bayesian optimization, implicit surface reconstruction, and terrain reconstruction. Fitting a GP to function values and derivatives at $n$ points in $d$ dimensions requires linear solves and log determinants with an ${n(d+1) \times n(d+1)}$ positive definite matrix -- leading to prohibitive $\mathcal{O}(n^3d^3)$ computations for standard direct methods. We propose iterative solvers using fast $\mathcal{O}(nd)$ matrix-vector multiplications (MVMs), together with pivoted Cholesky preconditioning that cuts the iterations to convergence by several orders of magnitude, allowing for fast kernel learning and prediction. Our approaches, together with dimensionality reduction, enables Bayesian optimization with derivatives to scale to high-dimensional problems and large evaluation budgets. △ Less

Submitted 29 October, 2018; originally announced October 2018.

Comments: Appears at Advances in Neural Information Processing Systems 32 (NIPS), 2018

Journal ref: Advances in Neural Information Processing Systems 32 (NIPS), 2018

BCOV invariants of Calabi--Yau manifolds and degenerations of Hodge structures

Authors: Dennis Eriksson, Gerard Freixas i Montplet, Christophe Mourougane

Abstract: Calabi--Yau manifolds have risen to prominence in algebraic geometry, in part because of mirror symmetry and enumerative geometry. After Bershadsky--Cecotti--Ooguri--Vafa (BCOV), it is expected that genus 1 curve counting on a Calabi--Yau manifold is related to a conjectured invariant, only depending on the complex structure of the mirror, and built from Ray--Singer holomorphic analytic torsions.… ▽ More Calabi--Yau manifolds have risen to prominence in algebraic geometry, in part because of mirror symmetry and enumerative geometry. After Bershadsky--Cecotti--Ooguri--Vafa (BCOV), it is expected that genus 1 curve counting on a Calabi--Yau manifold is related to a conjectured invariant, only depending on the complex structure of the mirror, and built from Ray--Singer holomorphic analytic torsions. To this end, extending work of Fang--Lu--Yoshikawa in dimension 3, we introduce and study the BCOV invariant of Calabi--Yau manifolds of arbitrary dimension. To determine it, knowledge of its behaviour at the boundary of moduli spaces is imperative. We address this problem by proving precise asymptotics along one-parameter degenerations, in terms of topological data and intersection theory. Central to the approach are new results on degenerations of $L^2$ metrics on Hodge bundles, combined with information on the singularities of Quillen metrics in our previous work. △ Less

Submitted 13 May, 2019; v1 submitted 14 September, 2018; originally announced September 2018.

Comments: Minor revision. Mainly restructure of the text, minor improvements and corrections. Added information about subdominant terms of $L^2$-norms

MSC Class: 14J32; 32G20; 58K55; 58J52 (Primary); 58K65 (Secondary)

Journal ref: Duke Math. J. 170, no. 3 (2021), 379-454

Spaces of norms, determinant of cohomology and Fekete points in non-Archimedean geometry

Authors: Sébastien Boucksom, Dennis Eriksson

Abstract: Let L be an ample line bundle on a (geometrically reduced) projective variety X over any complete valued field. Our main result describes the leading asymptotics of the determinant of cohomology of large powers of L, with respect to the supnorm of a continuous metric on the Berkovich analytification of L. As a consequence, we establish in this setting the existence of transfinite diameters and equ… ▽ More Let L be an ample line bundle on a (geometrically reduced) projective variety X over any complete valued field. Our main result describes the leading asymptotics of the determinant of cohomology of large powers of L, with respect to the supnorm of a continuous metric on the Berkovich analytification of L. As a consequence, we establish in this setting the existence of transfinite diameters and equidistribution of Fekete points, following a strategy going back Berman, Witt Nyström and the first author for complex manifolds. In the non-Archimedean case, our approach relies on a version of the Knudsen-Mumford expansion for the determinant of cohomology on models over the (possibly non-Noetherian) valuation ring, as a replacement for the asymptotic expansion of Bergman kernels in the complex case, and on the reduced fiber theorem, as a replacement for the Bernstein-Markov inequalities. Along the way, a systematic study of spaces of norms and the associated Fubini-Study type metrics is undertaken. △ Less

Submitted 8 January, 2021; v1 submitted 2 May, 2018; originally announced May 2018.

Comments: 99 pages. Final version, to appear in Advances Math

MSC Class: 32P05; 32U15

Scalable Log Determinants for Gaussian Process Kernel Learning

Authors: Kun Dong, David Eriksson, Hannes Nickisch, David Bindel, Andrew Gordon Wilson

Abstract: For applications as varied as Bayesian neural networks, determinantal point processes, elliptical graphical models, and kernel learning for Gaussian processes (GPs), one must compute a log determinant of an $n \times n$ positive definite matrix, and its derivatives - leading to prohibitive $\mathcal{O}(n^3)$ computations. We propose novel $\mathcal{O}(n)$ approaches to estimating these quantities… ▽ More For applications as varied as Bayesian neural networks, determinantal point processes, elliptical graphical models, and kernel learning for Gaussian processes (GPs), one must compute a log determinant of an $n \times n$ positive definite matrix, and its derivatives - leading to prohibitive $\mathcal{O}(n^3)$ computations. We propose novel $\mathcal{O}(n)$ approaches to estimating these quantities from only fast matrix vector multiplications (MVMs). These stochastic approximations are based on Chebyshev, Lanczos, and surrogate models, and converge quickly even for kernel matrices that have challenging spectra. We leverage these approximations to develop a scalable Gaussian process approach to kernel learning. We find that Lanczos is generally superior to Chebyshev for kernel learning, and that a surrogate approach can be highly efficient and accurate with popular kernels. △ Less

Submitted 9 November, 2017; originally announced November 2017.

Comments: Appears at Advances in Neural Information Processing Systems 30 (NIPS), 2017

Journal ref: Advances in Neural Information Processing Systems 30 (NIPS), 2017

Kähler quantization of vortex moduli

Authors: Dennis Eriksson, Nuno M. Romão

Abstract: We discuss the Kähler quantization of moduli spaces of vortices in line bundles over compact surfaces $Σ$. This furnishes a semiclassical framework for the study of quantum vortex dynamics in the Schrödinger-Chern-Simons model. We follow Deligne's approach to Quillen's metric in determinants of cohomology to construct all the quantum Hilbert spaces in this context. An alternative description of th… ▽ More We discuss the Kähler quantization of moduli spaces of vortices in line bundles over compact surfaces $Σ$. This furnishes a semiclassical framework for the study of quantum vortex dynamics in the Schrödinger-Chern-Simons model. We follow Deligne's approach to Quillen's metric in determinants of cohomology to construct all the quantum Hilbert spaces in this context. An alternative description of the quantum wavesections, in terms of multiparticle states of spinors on $Σ$ itself (valued in a prequantization of a multiple of its area form), is also obtained. This viewpoint sheds light on the nature of the quantum solitonic particles that emerge from the gauge theory. We find that in some cases (where the area of $Σ$ is small enough in relation to its genus) the dimensions of the quantum Hilbert spaces may be sensitive to the input data required by the quantization scheme, and also address the issue of relating different choices of such data geometrically. △ Less

Submitted 28 January, 2017; v1 submitted 27 December, 2016; originally announced December 2016.

Comments: 29 pages; v2 with small improvements

Journal ref: Lett. Math. Phys. 110 (2020) 659-693

Singularities of metrics on Hodge bundles and their topological invariants

Authors: Dennis Eriksson, Gerard Freixas i Montplet, Christophe Mourougane

Abstract: We consider degenerations of complex projective Calabi--Yau varieties and study the singularities of $L^2$, Quillen and BCOV metrics on Hodge and determinant bundles. The dominant and subdominant terms in the expansions of the metrics close to non-smooth fibers are shown to be related to well-known topological invariants of singularities, such as limit Hodge structures, vanishing cycles and log-ca… ▽ More We consider degenerations of complex projective Calabi--Yau varieties and study the singularities of $L^2$, Quillen and BCOV metrics on Hodge and determinant bundles. The dominant and subdominant terms in the expansions of the metrics close to non-smooth fibers are shown to be related to well-known topological invariants of singularities, such as limit Hodge structures, vanishing cycles and log-canonical thresholds. We also describe corresponding invariants for more general degenerating families in the case of the Quillen metric. △ Less

Submitted 9 November, 2016; originally announced November 2016.

MSC Class: 14J32; 58K55; 58J52 (Primary); 58K65; 14J70 (Secondary)

Journal ref: Algebraic Geometry 5 (6) (2018) 742-775

A logarithmic interpretation of Edixhoven's jumps for Jacobians

Authors: Dennis Eriksson, Lars Halvard Halle, Johannes Nicaise

Abstract: Let $A$ be an abelian variety over a discretely valued field. Edixhoven has defined a filtration on the special fiber of the Néron model of $A$ that measures the behaviour of the Néron model under tame base change. We interpret the jumps in this filtration in terms of lattices of logarithmic differential forms in the case where $A$ is the Jacobian of a curve $C$, and we give a compact explicit for… ▽ More Let $A$ be an abelian variety over a discretely valued field. Edixhoven has defined a filtration on the special fiber of the Néron model of $A$ that measures the behaviour of the Néron model under tame base change. We interpret the jumps in this filtration in terms of lattices of logarithmic differential forms in the case where $A$ is the Jacobian of a curve $C$, and we give a compact explicit formula for the jumps in terms of the combinatorial reduction data of $C$. △ Less

Submitted 21 March, 2014; originally announced March 2014.

The excess formula in functorial form

Authors: Dennis Eriksson

Abstract: This article is motivated by the need for better understanding of refined Riemann-Roch theorems and the behavior of the determinant of the cohomology. This poses a certain problem of functoriality and can be understood as that of giving refined constructions of operations in algebraic $K$-theory. In this article this is specialized to mean refining the excess formula, which measures the failure of… ▽ More This article is motivated by the need for better understanding of refined Riemann-Roch theorems and the behavior of the determinant of the cohomology. This poses a certain problem of functoriality and can be understood as that of giving refined constructions of operations in algebraic $K$-theory. In this article this is specialized to mean refining the excess formula, which measures the failure of base change, to the level of Deligne's virtual category. We give a natural set of properties for such a refinement, and prove that there exists a unique family of excess formulas on this refined level satisfying these properties. △ Less

Submitted 1 May, 2012; originally announced May 2012.

Discriminants and Artin conductors

Authors: Dennis Eriksson

Abstract: We study questions of multiplicities of discriminants for degenerations coming from projective duality over discrete valuation rings. The main result is a type of discriminant-different formula in the sense of classical algebraic number theory, and we relate it to Artin conductors via Bloch's conjecture. In the case of discriminants of planar curves we can calculate the different precisely. In gen… ▽ More We study questions of multiplicities of discriminants for degenerations coming from projective duality over discrete valuation rings. The main result is a type of discriminant-different formula in the sense of classical algebraic number theory, and we relate it to Artin conductors via Bloch's conjecture. In the case of discriminants of planar curves we can calculate the different precisely. In general these multiplicities encode topological invariants of the singular fibers and in the case of characteristic $p$, wild ramification data in the form of Swan conductors. △ Less

Submitted 15 June, 2011; originally announced June 2011.

Degenerating Riemann surfaces and the Quillen metric

Authors: Dennis Eriksson

Abstract: The degeneration of the Quillen metric for a one-parameter family of Riemann surfaces has been studied by Bismut-Bost and Yoshikawa. In this article we propose a more geometric point of view using Deligne's Riemann-Roch theorem. We obtain an interpretation of the singular part of the metric as a discriminant and the continuous part as a degeneration of the metric on Deligne products, which gives a… ▽ More The degeneration of the Quillen metric for a one-parameter family of Riemann surfaces has been studied by Bismut-Bost and Yoshikawa. In this article we propose a more geometric point of view using Deligne's Riemann-Roch theorem. We obtain an interpretation of the singular part of the metric as a discriminant and the continuous part as a degeneration of the metric on Deligne products, which gives an asymptotic development involving the monodromy eigenvalues. This generalizes the results of Bismut-Bost and is a version of Yoshikawa's results on the degeneration of the Quillen metric for general degenerations with isolated singularities in the central fiber. △ Less

Submitted 15 June, 2011; originally announced June 2011.

Interaction between gravitational waves and plasma waves in the Vlasov description

Authors: G. Brodin, M. Forsberg, M. Marklund, D. Eriksson

Abstract: The nonlinear interaction between electromagnetic, electrostatic and gravitational waves in a Vlasov plasma is reconsidered. By using a orthonormal tetrad description the three-wave coupling coefficients are computed. Comparing with previous results, it is found that the present theory leads to algebraic expression that are much reduced, as compared to those computed using a coordinate frame for… ▽ More The nonlinear interaction between electromagnetic, electrostatic and gravitational waves in a Vlasov plasma is reconsidered. By using a orthonormal tetrad description the three-wave coupling coefficients are computed. Comparing with previous results, it is found that the present theory leads to algebraic expression that are much reduced, as compared to those computed using a coordinate frame formalism. Furthermore, here we calculate the back-reaction on the gravitational waves, and a simple energy conservation law is deduced in the limit of a cold plasma. △ Less

Submitted 11 November, 2009; originally announced November 2009.

Comments: 9 pages, uses jpp.cls

A Deligne-Riemann-Roch isomorphism I: Preliminaries on virtual categories

Authors: Dennis Eriksson

Abstract: This is the first article in an upcoming series of papers. They have arisen through an attempt to answer open questions of Deligne proposed in "Le determinant de la cohomologie", Contemp. Mathematics 67 (1987). It amounts to functorial and metrized versions of the Grothendieck-Riemann-Roch theorem, as well as a Lefschetz-Riemann-Roch formula in the sense of Thomason, cf. "Lefschetz-Riemann-Roch… ▽ More This is the first article in an upcoming series of papers. They have arisen through an attempt to answer open questions of Deligne proposed in "Le determinant de la cohomologie", Contemp. Mathematics 67 (1987). It amounts to functorial and metrized versions of the Grothendieck-Riemann-Roch theorem, as well as a Lefschetz-Riemann-Roch formula in the sense of Thomason, cf. "Lefschetz-Riemann-Roch theorem and coherent trace formula" Ann. Sci. Ec. Norm. Sup. 18 (1985), Theorem 3.5. In this article we treat various preliminary results on virtual categories which are the categories where the Deligne-Riemann-Roch theorem is originally formulated. Finally we compare our constructions to constructions of Franke on Chern intersection functors and Chow categories. △ Less

Submitted 26 April, 2009; originally announced April 2009.

Comments: 71 pages

2HDMC - Two-Higgs-Doublet Model Calculator

Authors: David Eriksson, Johan Rathsman, Oscar Stål

Abstract: This manual describes the public code 2HDMC which can be used to perform calculations in a general, CP-conserving, two-Higgs-doublet model (2HDM). The program features simple conversion between different parametrizations of the 2HDM potential, a flexible Yukawa sector specification with choices of different Z_2-symmetries or more general couplings, a tree-level decay library including all two-bo… ▽ More This manual describes the public code 2HDMC which can be used to perform calculations in a general, CP-conserving, two-Higgs-doublet model (2HDM). The program features simple conversion between different parametrizations of the 2HDM potential, a flexible Yukawa sector specification with choices of different Z_2-symmetries or more general couplings, a tree-level decay library including all two-body - and some three-body - decay modes for the Higgs bosons, and the possibility to calculate observables of interest for constraining the 2HDM parameter space, as well as theoretical constraints from positivity and unitarity. The latest version of the 2HDMC code and full documentation is available from: http://www.isv.uu.se/thep/MC/2HDMC △ Less

Submitted 18 September, 2009; v1 submitted 5 February, 2009; originally announced February 2009.

Comments: 28 pages, no figures. Minor changes to the text, new interface to HiggsBounds described. Agrees with published version

Journal ref: Comput.Phys.Commun.181:189-205,2010

PYBBWH: A program for associated charged Higgs and W boson production

Authors: David Eriksson

Abstract: The Monte Carlo program, PYBBWH, is an implementation of the associated production of a charged Higgs and a W boson from bb fusion in a general Two-Higgs-Doublet model for both CP-conserving and CP-violating couplings. It is implemented as a external process to Pythia 6. The code can be downloaded from http://www.isv.uu.se/thep/MC/pybbwh/ The Monte Carlo program, PYBBWH, is an implementation of the associated production of a charged Higgs and a W boson from bb fusion in a general Two-Higgs-Doublet model for both CP-conserving and CP-violating couplings. It is implemented as a external process to Pythia 6. The code can be downloaded from http://www.isv.uu.se/thep/MC/pybbwh/ △ Less

Submitted 3 February, 2009; originally announced February 2009.

Comments: 5 pages, 1 figure; the code may be obtained from http://www.isv.uu.se/thep/MC/pybbwh/

Expected Performance of the ATLAS Experiment - Detector, Trigger and Physics

Authors: The ATLAS Collaboration, G. Aad, E. Abat, B. Abbott, J. Abdallah, A. A. Abdelalim, A. Abdesselam, O. Abdinov, B. Abi, M. Abolins, H. Abramowicz, B. S. Acharya, D. L. Adams, T. N. Addy, C. Adorisio, P. Adragna, T. Adye, J. A. Aguilar-Saavedra, M. Aharrouche, S. P. Ahlen, F. Ahles, A. Ahmad, H. Ahmed, G. Aielli, T. Akdogan , et al. (2587 additional authors not shown)

Abstract: A detailed study is presented of the expected performance of the ATLAS detector. The reconstruction of tracks, leptons, photons, missing energy and jets is investigated, together with the performance of b-tagging and the trigger. The physics potential for a variety of interesting physics processes, within the Standard Model and beyond, is examined. The study comprises a series of notes based on… ▽ More A detailed study is presented of the expected performance of the ATLAS detector. The reconstruction of tracks, leptons, photons, missing energy and jets is investigated, together with the performance of b-tagging and the trigger. The physics potential for a variety of interesting physics processes, within the Standard Model and beyond, is examined. The study comprises a series of notes based on simulations of the detector and physics processes, with particular emphasis given to the data expected from the first years of operation of the LHC at CERN. △ Less

Submitted 14 August, 2009; v1 submitted 28 December, 2008; originally announced January 2009.