-
Generalized Blaschke--Santaló-type inequalities, without symmetry restrictions
Authors:
Thomas A. Courtade,
Edric Wang
Abstract:
Nakamura and Tsuji (2024) recently investigated a many-function generalization of the functional Blaschke--Santaló inequality, which they refer to as a generalized Legendre duality relation. They showed that, among the class of all even test functions, centered Gaussian functions saturate this general family of functional inequalities. Leveraging a certain entropic duality, we give a short alterna…
▽ More
Nakamura and Tsuji (2024) recently investigated a many-function generalization of the functional Blaschke--Santaló inequality, which they refer to as a generalized Legendre duality relation. They showed that, among the class of all even test functions, centered Gaussian functions saturate this general family of functional inequalities. Leveraging a certain entropic duality, we give a short alternate proof of Nakamura and Tsuji's result, and, in the process, eliminate all symmetry assumptions. As an application, we establish a Talagrand-type inequality for the Wasserstein barycenter problem (without symmetry restrictions) originally conjectured by Kolesnikov and Werner (\textit{Adv.~Math.}, 2022). An analogous geometric Blaschke--Santaló-type inequality is established for many convex bodies, again without symmetry assumptions.
△ Less
Submitted 10 September, 2025;
originally announced September 2025.
-
Managing Correlations in Data and Privacy Demand
Authors:
Syomantak Chaudhuri,
Thomas A. Courtade
Abstract:
Previous works in the differential privacy literature that allow users to choose their privacy levels typically operate under the heterogeneous differential privacy (HDP) framework with the simplifying assumption that user data and privacy levels are not correlated. Firstly, we demonstrate that the standard HDP framework falls short when user data and privacy demands are allowed to be correlated.…
▽ More
Previous works in the differential privacy literature that allow users to choose their privacy levels typically operate under the heterogeneous differential privacy (HDP) framework with the simplifying assumption that user data and privacy levels are not correlated. Firstly, we demonstrate that the standard HDP framework falls short when user data and privacy demands are allowed to be correlated. Secondly, to address this shortcoming, we propose an alternate framework, Add-remove Heterogeneous Differential Privacy (AHDP), that jointly accounts for user data and privacy preference. We show that AHDP is robust to possible correlations between data and privacy. Thirdly, we formalize the guarantees of the proposed AHDP framework through an operational hypothesis testing perspective. The hypothesis testing setup may be of independent interest in analyzing other privacy frameworks as well. Fourthly, we show that there exists non-trivial AHDP mechanisms that notably do not require prior knowledge of the data-privacy correlations. We propose some such mechanisms and apply them to core statistical tasks such as mean estimation, frequency estimation, and linear regression. The proposed mechanisms are simple to implement with minimal assumptions and modeling requirements, making them attractive for real-world use. Finally, we empirically evaluate proposed AHDP mechanisms, highlighting their trade-offs using LLM-generated synthetic datasets, which we release for future research.
△ Less
Submitted 2 September, 2025;
originally announced September 2025.
-
Subadditivity of the log-Sobolev constant on convolutions
Authors:
Thomas A. Courtade,
Edric Wang
Abstract:
We present a general subadditivity inequality for log-Sobolev constants of convolution measures. As a corollary, we show that the log-Sobolev constant is monotone along the sequence of standardized convolutions in the central limit theorem.
We present a general subadditivity inequality for log-Sobolev constants of convolution measures. As a corollary, we show that the log-Sobolev constant is monotone along the sequence of standardized convolutions in the central limit theorem.
△ Less
Submitted 27 August, 2025;
originally announced August 2025.
-
Robust Estimation Under Heterogeneous Corruption Rates
Authors:
Syomantak Chaudhuri,
Jerry Li,
Thomas A. Courtade
Abstract:
We study the problem of robust estimation under heterogeneous corruption rates, where each sample may be independently corrupted with a known but non-identical probability. This setting arises naturally in distributed and federated learning, crowdsourcing, and sensor networks, yet existing robust estimators typically assume uniform or worst-case corruption, ignoring structural heterogeneity. For m…
▽ More
We study the problem of robust estimation under heterogeneous corruption rates, where each sample may be independently corrupted with a known but non-identical probability. This setting arises naturally in distributed and federated learning, crowdsourcing, and sensor networks, yet existing robust estimators typically assume uniform or worst-case corruption, ignoring structural heterogeneity. For mean estimation for multivariate bounded distributions and univariate gaussian distributions, we give tight minimax rates for all heterogeneous corruption patterns. For multivariate gaussian mean estimation and linear regression, we establish the minimax rate for squared error up to a factor of $\sqrt{d}$, where $d$ is the dimension. Roughly, our findings suggest that samples beyond a certain corruption threshold may be discarded by the optimal estimators -- this threshold is determined by the empirical distribution of the corruption rates given.
△ Less
Submitted 30 September, 2025; v1 submitted 20 August, 2025;
originally announced August 2025.
-
Online Assortment and Price Optimization Under Contextual Choice Models
Authors:
Yigit Efe Erginbas,
Thomas A. Courtade,
Kannan Ramchandran
Abstract:
We consider an assortment selection and pricing problem in which a seller has $N$ different items available for sale. In each round, the seller observes a $d$-dimensional contextual preference information vector for the user, and offers to the user an assortment of $K$ items at prices chosen by the seller. The user selects at most one of the products from the offered assortment according to a mult…
▽ More
We consider an assortment selection and pricing problem in which a seller has $N$ different items available for sale. In each round, the seller observes a $d$-dimensional contextual preference information vector for the user, and offers to the user an assortment of $K$ items at prices chosen by the seller. The user selects at most one of the products from the offered assortment according to a multinomial logit choice model whose parameters are unknown. The seller observes which, if any, item is chosen at the end of each round, with the goal of maximizing cumulative revenue over a selling horizon of length $T$. For this problem, we propose an algorithm that learns from user feedback and achieves a revenue regret of order $\widetilde{O}(d \sqrt{K T} / L_0 )$ where $L_0$ is the minimum price sensitivity parameter. We also obtain a lower bound of order $Ω(d \sqrt{T}/ L_0)$ for the regret achievable by any algorithm.
△ Less
Submitted 14 March, 2025;
originally announced March 2025.
-
Enhancing Feature-Specific Data Protection via Bayesian Coordinate Differential Privacy
Authors:
Maryam Aliakbarpour,
Syomantak Chaudhuri,
Thomas A. Courtade,
Alireza Fallah,
Michael I. Jordan
Abstract:
Local Differential Privacy (LDP) offers strong privacy guarantees without requiring users to trust external parties. However, LDP applies uniform protection to all data features, including less sensitive ones, which degrades performance of downstream tasks. To overcome this limitation, we propose a Bayesian framework, Bayesian Coordinate Differential Privacy (BCDP), that enables feature-specific p…
▽ More
Local Differential Privacy (LDP) offers strong privacy guarantees without requiring users to trust external parties. However, LDP applies uniform protection to all data features, including less sensitive ones, which degrades performance of downstream tasks. To overcome this limitation, we propose a Bayesian framework, Bayesian Coordinate Differential Privacy (BCDP), that enables feature-specific privacy quantification. This more nuanced approach complements LDP by adjusting privacy protection according to the sensitivity of each feature, enabling improved performance of downstream tasks without compromising privacy. We characterize the properties of BCDP and articulate its connections with standard non-Bayesian privacy frameworks. We further apply our BCDP framework to the problems of private mean estimation and ordinary least-squares regression. The BCDP-based approach obtains improved accuracy compared to a purely LDP-based approach, without compromising on privacy.
△ Less
Submitted 23 October, 2024;
originally announced October 2024.
-
Private Estimation when Data and Privacy Demands are Correlated
Authors:
Syomantak Chaudhuri,
Thomas A. Courtade
Abstract:
Differential Privacy (DP) is the current gold-standard for ensuring privacy for statistical queries. Estimation problems under DP constraints appearing in the literature have largely focused on providing equal privacy to all users. We consider the problems of empirical mean estimation for univariate data and frequency estimation for categorical data, both subject to heterogeneous privacy constrain…
▽ More
Differential Privacy (DP) is the current gold-standard for ensuring privacy for statistical queries. Estimation problems under DP constraints appearing in the literature have largely focused on providing equal privacy to all users. We consider the problems of empirical mean estimation for univariate data and frequency estimation for categorical data, both subject to heterogeneous privacy constraints. Each user, contributing a sample to the dataset, is allowed to have a different privacy demand. The dataset itself is assumed to be worst-case and we study both problems under two different formulations -- first, where privacy demands and data may be correlated, and second, where correlations are weakened by random permutation of the dataset. We establish theoretical performance guarantees for our proposed algorithms, under both PAC error and mean-squared error. These performance guarantees translate to minimax optimality in several instances, and experiments confirm superior performance of our algorithms over other baseline techniques.
△ Less
Submitted 18 April, 2025; v1 submitted 15 July, 2024;
originally announced July 2024.
-
Rigid characterizations of probability measures through independence, with applications
Authors:
Thomas A. Courtade
Abstract:
Three equivalent characterizations of probability measures through independence criteria are given. These characterizations lead to a family of Brascamp--Lieb-type inequalities for relative entropy, determine equilibrium states and sharp rates of convergence for certain linear Boltzmann-type dynamics, and unify an assortment of $L^2$ inequalities in probability.
Three equivalent characterizations of probability measures through independence criteria are given. These characterizations lead to a family of Brascamp--Lieb-type inequalities for relative entropy, determine equilibrium states and sharp rates of convergence for certain linear Boltzmann-type dynamics, and unify an assortment of $L^2$ inequalities in probability.
△ Less
Submitted 11 March, 2024;
originally announced March 2024.
-
Mean Estimation Under Heterogeneous Privacy Demands
Authors:
Syomantak Chaudhuri,
Konstantin Miagkov,
Thomas A. Courtade
Abstract:
Differential Privacy (DP) is a well-established framework to quantify privacy loss incurred by any algorithm. Traditional formulations impose a uniform privacy requirement for all users, which is often inconsistent with real-world scenarios in which users dictate their privacy preferences individually. This work considers the problem of mean estimation, where each user can impose their own distinc…
▽ More
Differential Privacy (DP) is a well-established framework to quantify privacy loss incurred by any algorithm. Traditional formulations impose a uniform privacy requirement for all users, which is often inconsistent with real-world scenarios in which users dictate their privacy preferences individually. This work considers the problem of mean estimation, where each user can impose their own distinct privacy level. The algorithm we propose is shown to be minimax optimal and has a near-linear run-time. Our results elicit an interesting saturation phenomenon that occurs. Namely, the privacy requirements of the most stringent users dictate the overall error rates. As a consequence, users with less but differing privacy requirements are all given more privacy than they require, in equal amounts. In other words, these privacy-indifferent users are given a nontrivial degree of privacy for free, without any sacrifice in the performance of the estimator.
△ Less
Submitted 19 October, 2023;
originally announced October 2023.
-
Mean Estimation Under Heterogeneous Privacy: Some Privacy Can Be Free
Authors:
Syomantak Chaudhuri,
Thomas A. Courtade
Abstract:
Differential Privacy (DP) is a well-established framework to quantify privacy loss incurred by any algorithm. Traditional DP formulations impose a uniform privacy requirement for all users, which is often inconsistent with real-world scenarios in which users dictate their privacy preferences individually. This work considers the problem of mean estimation under heterogeneous DP constraints, where…
▽ More
Differential Privacy (DP) is a well-established framework to quantify privacy loss incurred by any algorithm. Traditional DP formulations impose a uniform privacy requirement for all users, which is often inconsistent with real-world scenarios in which users dictate their privacy preferences individually. This work considers the problem of mean estimation under heterogeneous DP constraints, where each user can impose their own distinct privacy level. The algorithm we propose is shown to be minimax optimal when there are two groups of users with distinct privacy levels. Our results elicit an interesting saturation phenomenon that occurs as one group's privacy level is relaxed, while the other group's privacy level remains constant. Namely, after a certain point, further relaxing the privacy requirement of the former group does not improve the performance of the minimax optimal mean estimator. Thus, the central server can offer a certain degree of privacy without any sacrifice in performance.
△ Less
Submitted 27 April, 2023;
originally announced May 2023.
-
Entropy Inequalities and Gaussian Comparisons
Authors:
Efe Aras,
Thomas A. Courtade
Abstract:
We establish a general class of entropy inequalities that take the concise form of Gaussian comparisons. The main result unifies many classical and recent results, including the Shannon-Stam inequality, the Brunn-Minkowski inequality, the Zamir-Feder inequality, the Brascamp-Lieb and Barthe inequalities, the Anantharam-Jog-Nair inequality, and others.
We establish a general class of entropy inequalities that take the concise form of Gaussian comparisons. The main result unifies many classical and recent results, including the Shannon-Stam inequality, the Brunn-Minkowski inequality, the Zamir-Feder inequality, the Brascamp-Lieb and Barthe inequalities, the Anantharam-Jog-Nair inequality, and others.
△ Less
Submitted 28 June, 2022;
originally announced June 2022.
-
Equality cases in the Anantharam-Jog-Nair inequality
Authors:
Efe Aras,
Thomas A. Courtade,
Albert Zhang
Abstract:
Anantharam, Jog and Nair recently unified the Shannon-Stam inequality and the entropic form of the Brascamp-Lieb inequalities under a common inequality. They left open the problems of extremizability and characterization of extremizers. Both questions are resolved in the present paper.
Anantharam, Jog and Nair recently unified the Shannon-Stam inequality and the entropic form of the Brascamp-Lieb inequalities under a common inequality. They left open the problems of extremizability and characterization of extremizers. Both questions are resolved in the present paper.
△ Less
Submitted 23 June, 2022;
originally announced June 2022.
-
Linear Models are Most Favorable among Generalized Linear Models
Authors:
Kuan-Yun Lee,
Thomas A. Courtade
Abstract:
We establish a nonasymptotic lower bound on the $L_2$ minimax risk for a class of generalized linear models. It is further shown that the minimax risk for the canonical linear model matches this lower bound up to a universal constant. Therefore, the canonical linear model may be regarded as most favorable among the considered class of generalized linear models (in terms of minimax risk). The proof…
▽ More
We establish a nonasymptotic lower bound on the $L_2$ minimax risk for a class of generalized linear models. It is further shown that the minimax risk for the canonical linear model matches this lower bound up to a universal constant. Therefore, the canonical linear model may be regarded as most favorable among the considered class of generalized linear models (in terms of minimax risk). The proof makes use of an information-theoretic Bayesian Cramér-Rao bound for log-concave priors, established by Aras et al. (2019).
△ Less
Submitted 9 June, 2020;
originally announced June 2020.
-
Euclidean Forward-Reverse Brascamp-Lieb Inequalities: Finiteness, Structure and Extremals
Authors:
Thomas A. Courtade,
Jingbo Liu
Abstract:
A new proof is given for the fact that centered gaussian functions saturate the Euclidean forward-reverse Brascamp-Lieb inequalities, extending the Brascamp-Lieb and Barthe theorems. A duality principle for best constants is also developed, which generalizes the fact that the best constants in the Brascamp-Lieb and Barthe inequalities are equal. Finally, as the title hints, the main results concer…
▽ More
A new proof is given for the fact that centered gaussian functions saturate the Euclidean forward-reverse Brascamp-Lieb inequalities, extending the Brascamp-Lieb and Barthe theorems. A duality principle for best constants is also developed, which generalizes the fact that the best constants in the Brascamp-Lieb and Barthe inequalities are equal. Finally, as the title hints, the main results concerning finiteness, structure and gaussian-extremizability for the Brascamp-Lieb inequality due to Bennett, Carbery, Christ and Tao are generalized to the setting of the forward-reverse Brascamp-Lieb inequality.
△ Less
Submitted 29 August, 2019; v1 submitted 29 July, 2019;
originally announced July 2019.
-
A Family of Bayesian Cramér-Rao Bounds, and Consequences for Log-Concave Priors
Authors:
Efe Aras,
Kuan-Yun Lee,
Ashwin Pananjady,
Thomas A. Courtade
Abstract:
Under minimal regularity assumptions, we establish a family of information-theoretic Bayesian Cramér-Rao bounds, indexed by probability measures that satisfy a logarithmic Sobolev inequality. This family includes as a special case the known Bayesian Cramér-Rao bound (or van Trees inequality), and its less widely known entropic improvement due to Efroimovich. For the setting of a log-concave prior,…
▽ More
Under minimal regularity assumptions, we establish a family of information-theoretic Bayesian Cramér-Rao bounds, indexed by probability measures that satisfy a logarithmic Sobolev inequality. This family includes as a special case the known Bayesian Cramér-Rao bound (or van Trees inequality), and its less widely known entropic improvement due to Efroimovich. For the setting of a log-concave prior, we obtain a Bayesian Cramér-Rao bound which holds for any (possibly biased) estimator and, unlike the van Trees inequality, does not depend on the Fisher information of the prior.
△ Less
Submitted 22 February, 2019;
originally announced February 2019.
-
Transportation Proof of an inequality by Anantharam, Jog and Nair
Authors:
Thomas A. Courtade
Abstract:
Anantharam, Jog and Nair recently put forth an entropic inequality which simultaneously generalizes the Shannon-Stam entropy power inequality and the Brascamp-Lieb inequality in entropic form. We give a brief proof of their result based on optimal transport.
Anantharam, Jog and Nair recently put forth an entropic inequality which simultaneously generalizes the Shannon-Stam entropy power inequality and the Brascamp-Lieb inequality in entropic form. We give a brief proof of their result based on optimal transport.
△ Less
Submitted 31 January, 2019; v1 submitted 30 January, 2019;
originally announced January 2019.
-
Bounds on the Poincaré constant for convolution measures
Authors:
Thomas A. Courtade
Abstract:
We establish a Shearer-type inequality for the Poincaré constant, showing that the Poincaré constant corresponding to the convolution of a collection of measures can be nontrivially controlled by the Poincaré constants corresponding to convolutions of subsets of measures. This implies, for example, that the Poincaré constant is non-increasing along the central limit theorem. We also establish a di…
▽ More
We establish a Shearer-type inequality for the Poincaré constant, showing that the Poincaré constant corresponding to the convolution of a collection of measures can be nontrivially controlled by the Poincaré constants corresponding to convolutions of subsets of measures. This implies, for example, that the Poincaré constant is non-increasing along the central limit theorem. We also establish a dimension-free stability estimate for subadditivity of the Poincaré constant on convolutions which uniformly improves an earlier one-dimensional estimate of a similar nature by Johnson (2004). As a byproduct of our arguments, we find that the monotone properties of entropy, Fisher information and the Poincaré constant along the CLT find a common root in Shearer's inequality.
△ Less
Submitted 29 June, 2018;
originally announced July 2018.
-
Worst-case vs Average-case Design for Estimation from Fixed Pairwise Comparisons
Authors:
Ashwin Pananjady,
Cheng Mao,
Vidya Muthukumar,
Martin J. Wainwright,
Thomas A. Courtade
Abstract:
Pairwise comparison data arises in many domains, including tournament rankings, web search, and preference elicitation. Given noisy comparisons of a fixed subset of pairs of items, we study the problem of estimating the underlying comparison probabilities under the assumption of strong stochastic transitivity (SST). We also consider the noisy sorting subclass of the SST model. We show that when th…
▽ More
Pairwise comparison data arises in many domains, including tournament rankings, web search, and preference elicitation. Given noisy comparisons of a fixed subset of pairs of items, we study the problem of estimating the underlying comparison probabilities under the assumption of strong stochastic transitivity (SST). We also consider the noisy sorting subclass of the SST model. We show that when the assignment of items to the topology is arbitrary, these permutation-based models, unlike their parametric counterparts, do not admit consistent estimation for most comparison topologies used in practice. We then demonstrate that consistent estimation is possible when the assignment of items to the topology is randomized, thus establishing a dichotomy between worst-case and average-case designs. We propose two estimators in the average-case setting and analyze their risk, showing that it depends on the comparison topology only through the degree sequence of the topology. The rates achieved by these estimators are shown to be optimal for a large class of graphs. Our results are corroborated by simulations on multiple comparison topologies.
△ Less
Submitted 19 July, 2017;
originally announced July 2017.
-
Denoising Linear Models with Permuted Data
Authors:
Ashwin Pananjady,
Martin J. Wainwright,
Thomas A. Courtade
Abstract:
The multivariate linear regression model with shuffled data and additive Gaussian noise arises in various correspondence estimation and matching problems. Focusing on the denoising aspect of this problem, we provide a characterization the minimax error rate that is sharp up to logarithmic factors. We also analyze the performance of two versions of a computationally efficient estimator, and establi…
▽ More
The multivariate linear regression model with shuffled data and additive Gaussian noise arises in various correspondence estimation and matching problems. Focusing on the denoising aspect of this problem, we provide a characterization the minimax error rate that is sharp up to logarithmic factors. We also analyze the performance of two versions of a computationally efficient estimator, and establish their consistency for a large range of input parameters. Finally, we provide an exact algorithm for the noiseless problem and demonstrate its performance on an image point-cloud matching task. Our analysis also extends to datasets with outliers.
△ Less
Submitted 24 April, 2017;
originally announced April 2017.
-
A Counterexample to the Vector Generalization of Costa's EPI, and Partial Resolution
Authors:
Thomas A. Courtade,
Guangyue Han,
Yaochen Wu
Abstract:
We give a counterexample to the vector generalization of Costa's entropy power inequality (EPI) due to Liu, Liu, Poor and Shamai. In particular, the claimed inequality can fail if the matix-valued parameter in the convex combination does not commute with the covariance of the additive Gaussian noise. Conversely, the inequality holds if these two matrices commute.
We give a counterexample to the vector generalization of Costa's entropy power inequality (EPI) due to Liu, Liu, Poor and Shamai. In particular, the claimed inequality can fail if the matix-valued parameter in the convex combination does not commute with the covariance of the additive Gaussian noise. Conversely, the inequality holds if these two matrices commute.
△ Less
Submitted 20 April, 2017;
originally announced April 2017.
-
Existence of Stein Kernels under a Spectral Gap, and Discrepancy Bound
Authors:
Thomas A. Courtade,
Max Fathi,
Ashwin Pananjady
Abstract:
We establish existence of Stein kernels for probability measures on $\mathbb{R}^d$ satisfying a Poincaré inequality, and obtain bounds on the Stein discrepancy of such measures. Applications to quantitative central limit theorems are discussed, including a new CLT in Wasserstein distance $W_2$ with optimal rate and dependence on the dimension. As a byproduct, we obtain a stability version of an es…
▽ More
We establish existence of Stein kernels for probability measures on $\mathbb{R}^d$ satisfying a Poincaré inequality, and obtain bounds on the Stein discrepancy of such measures. Applications to quantitative central limit theorems are discussed, including a new CLT in Wasserstein distance $W_2$ with optimal rate and dependence on the dimension. As a byproduct, we obtain a stability version of an estimate of the Poincaré constant of probability measures under a second moment constraint. The results extend more generally to the setting of converse weighted Poincaré inequalities. The proof is based on simple arguments of calculus of variations.
Further, we establish two general properties enjoyed by the Stein discrepancy, holding whenever a Stein kernel exists: Stein discrepancy is strictly decreasing along the CLT, and it controls the skewness of a random vector.
△ Less
Submitted 8 March, 2018; v1 submitted 22 March, 2017;
originally announced March 2017.
-
Information-Theoretic Perspectives on Brascamp-Lieb Inequality and Its Reverse
Authors:
Jingbo Liu,
Thomas A. Courtade,
Paul Cuff,
Sergio Verdu
Abstract:
We introduce an inequality which may be viewed as a generalization of both the Brascamp-Lieb inequality and its reverse (Barthe's inequality), and prove its information-theoretic (i.e.\ entropic) formulation. This result leads to a unified approach to functional inequalities such as the variational formula of Rényi entropy, hypercontractivity and its reverse, strong data processing inequalities, a…
▽ More
We introduce an inequality which may be viewed as a generalization of both the Brascamp-Lieb inequality and its reverse (Barthe's inequality), and prove its information-theoretic (i.e.\ entropic) formulation. This result leads to a unified approach to functional inequalities such as the variational formula of Rényi entropy, hypercontractivity and its reverse, strong data processing inequalities, and transportation-cost inequalities, whose utility in the proofs of various coding theorems has gained growing popularity recently. We show that our information-theoretic setting is convenient for proving properties such as data processing, tensorization, convexity (Riesz-Thorin interpolation) and Gaussian optimality. In particular, we elaborate on a "doubling trick" used by Lieb and Geng-Nair to prove several results on Gaussian optimality. Several applications are discussed, including a generalization of the Brascamp-Lieb inequality involving Gaussian random transformations, the determination of Wyner's common information of vector Gaussian sources, and the achievable rate region of certain key generation problems in the case of vector Gaussian sources.
△ Less
Submitted 3 December, 2017; v1 submitted 20 February, 2017;
originally announced February 2017.
-
Wasserstein Stability of the Entropy Power Inequality for Log-Concave Densities
Authors:
Thomas A. Courtade,
Max Fathi,
Ashwin Pananjady
Abstract:
We establish quantitative stability results for the entropy power inequality (EPI). Specifically, we show that if uniformly log-concave densities nearly saturate the EPI, then they must be close to Gaussian densities in the quadratic Wasserstein distance. Further, if one of the densities is log-concave and the other is Gaussian, then the deficit in the EPI can be controlled in terms of the $L^1$-W…
▽ More
We establish quantitative stability results for the entropy power inequality (EPI). Specifically, we show that if uniformly log-concave densities nearly saturate the EPI, then they must be close to Gaussian densities in the quadratic Wasserstein distance. Further, if one of the densities is log-concave and the other is Gaussian, then the deficit in the EPI can be controlled in terms of the $L^1$-Wasserstein distance. As a counterpoint, an example shows that the EPI can be unstable with respect to the quadratic Wasserstein distance when densities are uniformly log-concave on sets of measure arbitrarily close to one. Our stability results can be extended to non-log-concave densities, provided certain regularity conditions are met. The proofs are based on optimal transportation.
△ Less
Submitted 25 October, 2016;
originally announced October 2016.
-
Monotonicity of Entropy and Fisher Information: A Quick Proof via Maximal Correlation
Authors:
Thomas A. Courtade
Abstract:
A simple proof is given for the monotonicity of entropy and Fisher information associated to sums of i.i.d. random variables. The proof relies on a characterization of maximal correlation for partial sums due to Dembo, Kagan and Shepp.
A simple proof is given for the monotonicity of entropy and Fisher information associated to sums of i.i.d. random variables. The proof relies on a characterization of maximal correlation for partial sums due to Dembo, Kagan and Shepp.
△ Less
Submitted 13 October, 2016;
originally announced October 2016.
-
Links between the Logarithmic Sobolev Inequality and the convolution inequalities for Entropy and Fisher Information
Authors:
Thomas A. Courtade
Abstract:
Relative to the Gaussian measure on $\mathbb{R}^d$, entropy and Fisher information are famously related via Gross' logarithmic Sobolev inequality (LSI). These same functionals also separately satisfy convolution inequalities, as proved by Stam. We establish a dimension-free inequality that interpolates among these relations. Several interesting corollaries follow: (i) the deficit in the LSI satisf…
▽ More
Relative to the Gaussian measure on $\mathbb{R}^d$, entropy and Fisher information are famously related via Gross' logarithmic Sobolev inequality (LSI). These same functionals also separately satisfy convolution inequalities, as proved by Stam. We establish a dimension-free inequality that interpolates among these relations. Several interesting corollaries follow: (i) the deficit in the LSI satisfies a convolution inequality itself; (ii) the deficit in the LSI controls convergence in the entropic and Fisher information central limit theorems; and (iii) the LSI is stable with respect to HWI jumps (i.e., a jump in any of the convolution inequalities associated to the HWI functionals).
Another consequence is that the convolution inequalities for Fisher information and entropy powers are reversible in general, up to a factor depending on the Stam defect. An improved form of Nelson's hypercontractivity estimate also follows. Finally, we speculate on the possibility of an analogous reverse Brunn-Minkowski inequality and a related upper bound on surface area associated to Minkowski sums.
△ Less
Submitted 18 August, 2016;
originally announced August 2016.
-
Entropy Jumps for Radially Symmetric Random Vectors
Authors:
Thomas A. Courtade
Abstract:
We establish a quantitative bound on the entropy jump associated to the sum of independent, identically distributed (IID) radially symmetric random vectors having dimension greater than one. Following the usual approach, we first consider the analogous problem of Fisher information dissipation, and then integrate along the Ornstein-Uhlenbeck semigroup to obtain an entropic inequality. In a departu…
▽ More
We establish a quantitative bound on the entropy jump associated to the sum of independent, identically distributed (IID) radially symmetric random vectors having dimension greater than one. Following the usual approach, we first consider the analogous problem of Fisher information dissipation, and then integrate along the Ornstein-Uhlenbeck semigroup to obtain an entropic inequality. In a departure from previous work, we appeal to a result by Desvillettes and Villani on entropy production associated to the Landau equation. This obviates strong regularity assumptions, such as presence of a spectral gap and log-concavity of densities, but comes at the expense of radial symmetry. As an application, we give a quantitative estimate of the deficit in the Gaussian logarithmic Sobolev inequality for radially symmetric functions.
△ Less
Submitted 3 November, 2016; v1 submitted 18 August, 2016;
originally announced August 2016.
-
Linear Regression with an Unknown Permutation: Statistical and Computational Limits
Authors:
Ashwin Pananjady,
Martin J. Wainwright,
Thomas A. Courtade
Abstract:
Consider a noisy linear observation model with an unknown permutation, based on observing $y = Π^* A x^* + w$, where $x^* \in \mathbb{R}^d$ is an unknown vector, $Π^*$ is an unknown $n \times n$ permutation matrix, and $w \in \mathbb{R}^n$ is additive Gaussian noise. We analyze the problem of permutation recovery in a random design setting in which the entries of the matrix $A$ are drawn i.i.d. fr…
▽ More
Consider a noisy linear observation model with an unknown permutation, based on observing $y = Π^* A x^* + w$, where $x^* \in \mathbb{R}^d$ is an unknown vector, $Π^*$ is an unknown $n \times n$ permutation matrix, and $w \in \mathbb{R}^n$ is additive Gaussian noise. We analyze the problem of permutation recovery in a random design setting in which the entries of the matrix $A$ are drawn i.i.d. from a standard Gaussian distribution, and establish sharp conditions on the SNR, sample size $n$, and dimension $d$ under which $Π^*$ is exactly and approximately recoverable. On the computational front, we show that the maximum likelihood estimate of $Π^*$ is NP-hard to compute, while also providing a polynomial time algorithm when $d =1$.
△ Less
Submitted 9 August, 2016;
originally announced August 2016.
-
Brascamp-Lieb Inequality and Its Reverse: An Information Theoretic View
Authors:
Jingbo Liu,
Thomas A. Courtade,
Paul Cuff,
Sergio Verdu
Abstract:
We generalize a result by Carlen and Cordero-Erausquin on the equivalence between the Brascamp-Lieb inequality and the subadditivity of relative entropy by allowing for random transformations (a broadcast channel). This leads to a unified perspective on several functional inequalities that have been gaining popularity in the context of proving impossibility results. We demonstrate that the informa…
▽ More
We generalize a result by Carlen and Cordero-Erausquin on the equivalence between the Brascamp-Lieb inequality and the subadditivity of relative entropy by allowing for random transformations (a broadcast channel). This leads to a unified perspective on several functional inequalities that have been gaining popularity in the context of proving impossibility results. We demonstrate that the information theoretic dual of the Brascamp-Lieb inequality is a convenient setting for proving properties such as data processing, tensorization, convexity and Gaussian optimality. Consequences of the latter include an extension of the Brascamp-Lieb inequality allowing for Gaussian random transformations, the determination of the multivariate Wyner common information for Gaussian sources, and a multivariate version of Nelson's hypercontractivity theorem. Finally we present an information theoretic characterization of a reverse Brascamp-Lieb inequality involving a random transformation (a multiple access channel).
△ Less
Submitted 9 May, 2016;
originally announced May 2016.
-
Partial DNA Assembly: A Rate-Distortion Perspective
Authors:
Ilan Shomorony,
Govinda M. Kamath,
Fei Xia,
Thomas A. Courtade,
David N. Tse
Abstract:
Earlier formulations of the DNA assembly problem were all in the context of perfect assembly; i.e., given a set of reads from a long genome sequence, is it possible to perfectly reconstruct the original sequence? In practice, however, it is very often the case that the read data is not sufficiently rich to permit unambiguous reconstruction of the original sequence. While a natural generalization o…
▽ More
Earlier formulations of the DNA assembly problem were all in the context of perfect assembly; i.e., given a set of reads from a long genome sequence, is it possible to perfectly reconstruct the original sequence? In practice, however, it is very often the case that the read data is not sufficiently rich to permit unambiguous reconstruction of the original sequence. While a natural generalization of the perfect assembly formulation to these cases would be to consider a rate-distortion framework, partial assemblies are usually represented in terms of an assembly graph, making the definition of a distortion measure challenging. In this work, we introduce a distortion function for assembly graphs that can be understood as the logarithm of the number of Eulerian cycles in the assembly graph, each of which correspond to a candidate assembly that could have generated the observed reads. We also introduce an algorithm for the construction of an assembly graph and analyze its performance on real genomes.
△ Less
Submitted 6 May, 2016;
originally announced May 2016.
-
Strengthening the Entropy Power Inequality
Authors:
Thomas A. Courtade
Abstract:
We tighten the Entropy Power Inequality (EPI) when one of the random summands is Gaussian. Our strengthening is closely connected to the concept of strong data processing for Gaussian channels and generalizes the (vector extension of) Costa's EPI. This leads to a new reverse entropy power inequality and, as a corollary, sharpens Stam's inequality relating entropy power and Fisher information. Appl…
▽ More
We tighten the Entropy Power Inequality (EPI) when one of the random summands is Gaussian. Our strengthening is closely connected to the concept of strong data processing for Gaussian channels and generalizes the (vector extension of) Costa's EPI. This leads to a new reverse entropy power inequality and, as a corollary, sharpens Stam's inequality relating entropy power and Fisher information. Applications to network information theory are given, including a short self-contained proof of the rate region for the two-encoder quadratic Gaussian source coding problem.
Our argument is based on weak convergence and a technique employed by Geng and Nair for establishing Gaussian optimality via rotational-invariance, which traces its roots to a `doubling trick' that has been successfully used in the study of functional inequalities.
△ Less
Submitted 9 February, 2016;
originally announced February 2016.
-
Smoothing Brascamp-Lieb Inequalities and Strong Converses for Common Randomness Generation
Authors:
Jingbo Liu,
Thomas A. Courtade,
Paul Cuff,
Sergio Verdu
Abstract:
We study the infimum of the best constant in a functional inequality, the Brascamp-Lieb-like inequality, over auxiliary measures within a neighborhood of a product distribution. In the finite alphabet and the Gaussian cases, such an infimum converges to the best constant in a mutual information inequality. Implications for strong converse properties of two common randomness (CR) generation problem…
▽ More
We study the infimum of the best constant in a functional inequality, the Brascamp-Lieb-like inequality, over auxiliary measures within a neighborhood of a product distribution. In the finite alphabet and the Gaussian cases, such an infimum converges to the best constant in a mutual information inequality. Implications for strong converse properties of two common randomness (CR) generation problems are discussed. In particular, we prove the strong converse property of the rate region for the omniscient helper CR generation problem in the discrete and the Gaussian cases. The latter case is perhaps the first instance of a strong converse for a continuous source when the rate region involves auxiliary random variables.
△ Less
Submitted 6 February, 2016;
originally announced February 2016.
-
Compressing Sparse Sequences under Local Decodability Constraints
Authors:
Ashwin Pananjady,
Thomas A. Courtade
Abstract:
We consider a variable-length source coding problem subject to local decodability constraints. In particular, we investigate the blocklength scaling behavior attainable by encodings of $r$-sparse binary sequences, under the constraint that any source bit can be correctly decoded upon probing at most $d$ codeword bits. We consider both adaptive and non-adaptive access models, and derive upper and l…
▽ More
We consider a variable-length source coding problem subject to local decodability constraints. In particular, we investigate the blocklength scaling behavior attainable by encodings of $r$-sparse binary sequences, under the constraint that any source bit can be correctly decoded upon probing at most $d$ codeword bits. We consider both adaptive and non-adaptive access models, and derive upper and lower bounds that often coincide up to constant factors. Notably, such a characterization for the fixed-blocklength analog of our problem remains unknown, despite considerable research over the last three decades. Connections to communication complexity are also briefly discussed.
△ Less
Submitted 8 April, 2015;
originally announced April 2015.
-
Coded Cooperative Data Exchange for a Secret Key
Authors:
Thomas A. Courtade,
Thomas R. Halford
Abstract:
We consider a coded cooperative data exchange problem with the goal of generating a secret key. Specifically, we investigate the number of public transmissions required for a set of clients to agree on a secret key with probability one, subject to the constraint that it remains private from an eavesdropper.
Although the problems are closely related, we prove that secret key generation with fewes…
▽ More
We consider a coded cooperative data exchange problem with the goal of generating a secret key. Specifically, we investigate the number of public transmissions required for a set of clients to agree on a secret key with probability one, subject to the constraint that it remains private from an eavesdropper.
Although the problems are closely related, we prove that secret key generation with fewest number of linear transmissions is NP-hard, while it is known that the analogous problem in traditional cooperative data exchange can be solved in polynomial time. In doing this, we completely characterize the best possible performance of linear coding schemes, and also prove that linear codes can be strictly suboptimal. Finally, we extend the single-key results to characterize the minimum number of public transmissions required to generate a desired integer number of statistically independent secret keys.
△ Less
Submitted 1 July, 2014;
originally announced July 2014.
-
Outer Bounds for Multiterminal Source Coding via a Strong Data Processing Inequality
Authors:
Thomas A. Courtade
Abstract:
An intuitive outer bound for the multiterminal source coding problem is given. The proposed bound explicitly couples the rate distortion functions for each source and correlation measures which derive from a "strong" data processing inequality. Unlike many standard outer bounds, the proposed bound is not parameterized by a continuous family of auxiliary random variables, but instead only requires…
▽ More
An intuitive outer bound for the multiterminal source coding problem is given. The proposed bound explicitly couples the rate distortion functions for each source and correlation measures which derive from a "strong" data processing inequality. Unlike many standard outer bounds, the proposed bound is not parameterized by a continuous family of auxiliary random variables, but instead only requires maximizing two ratios of divergences which do not depend on the distortion functions under consideration.
△ Less
Submitted 15 July, 2013; v1 submitted 14 February, 2013;
originally announced February 2013.
-
Which Boolean Functions are Most Informative?
Authors:
Gowtham R. Kumar,
Thomas A. Courtade
Abstract:
We introduce a simply stated conjecture regarding the maximum mutual information a Boolean function can reveal about noisy inputs. Specifically, let $X^n$ be i.i.d. Bernoulli(1/2), and let $Y^n$ be the result of passing $X^n$ through a memoryless binary symmetric channel with crossover probability $α$. For any Boolean function $b:\{0,1\}^n\rightarrow \{0,1\}$, we conjecture that…
▽ More
We introduce a simply stated conjecture regarding the maximum mutual information a Boolean function can reveal about noisy inputs. Specifically, let $X^n$ be i.i.d. Bernoulli(1/2), and let $Y^n$ be the result of passing $X^n$ through a memoryless binary symmetric channel with crossover probability $α$. For any Boolean function $b:\{0,1\}^n\rightarrow \{0,1\}$, we conjecture that $I(b(X^n);Y^n)\leq 1-H(α)$. While the conjecture remains open, we provide substantial evidence supporting its validity.
△ Less
Submitted 15 July, 2013; v1 submitted 11 February, 2013;
originally announced February 2013.
-
Coded Cooperative Data Exchange in Multihop Networks
Authors:
Thomas A. Courtade,
Richard D. Wesel
Abstract:
Consider a connected network of n nodes that all wish to recover k desired packets. Each node begins with a subset of the desired packets and exchanges coded packets with its neighbors. This paper provides necessary and sufficient conditions which characterize the set of all transmission schemes that permit every node to ultimately learn (recover) all k packets. When the network satisfies certain…
▽ More
Consider a connected network of n nodes that all wish to recover k desired packets. Each node begins with a subset of the desired packets and exchanges coded packets with its neighbors. This paper provides necessary and sufficient conditions which characterize the set of all transmission schemes that permit every node to ultimately learn (recover) all k packets. When the network satisfies certain regularity conditions and packets are randomly distributed, this paper provides tight concentration results on the number of transmissions required to achieve universal recovery. For the case of a fully connected network, a polynomial-time algorithm for computing an optimal transmission scheme is derived. An application to secrecy generation is discussed.
△ Less
Submitted 15 March, 2012;
originally announced March 2012.