2022/05/12 by Zieder, Ian, Dytso, Alex, Cardone, Martina
#FOS: Computer and information sciences #FOS: Electrical engineering #Information Theory (cs.IT) #Machine Learning (stat.ML) #Signal Processing (eess.SP) #electronic engineering #information engineering
paper · doi:10.48550/arxiv.2205.05848
This paper studies the minimum mean squared error (MMSE) of estimating X ∈ ℝd from the noisy observation Y ∈ ℝk, under the assumption that the noise (i.e., Y|X) is a member of the exponential family. The paper provides a new lower bound on the MMSE. Towards this end, an alternative representation of the MMSE is first presented, which is argued to be useful in deriving closed-form expressions for the MMSE. This new representation is then used together with the Poincaré inequality to provide a new lower bound on the MMSE. Unlike, for example, the Cramér-Rao bound, the new bound holds for all possible distributions on the input X. Moreover, the lower bound is shown to be tight in the high-noise regime for the Gaussian noise setting under the assumption that X is sub-Gaussian. Finally, several numerical examples are shown which demonstrate that the bound performs well in all noise regimes.