1995/12/01 by David H. Wolpert, David R. Wolf · 5 citations
Computer Science · Engineering · Physics and Astronomy · Mathematics · #Neural Networks and Applications #Control Systems and Identification #Statistical Mechanics and Entropy #Estimator #Mathematics #Bayes' theorem #Applied mathematics #Bayes estimator #Probability distribution #Entropy (arrow of time) #Prior probability #Principle of maximum entropy #Statistics #Covariance #Finite set #Bayesian probability #Statistical physics #Mathematical analysis #Physics #Quantum mechanics
paper · doi:10.1103/physreve.52.6841
openalex publication_date 1995/12/01 · openalex created_date 2020/05/01 · openalex updated_date 2026/07/15
This paper addresses the problem of estimating a function of a probability distribution from a finite set of samples of that distribution. A Bayesian analysis of this problem is presented, the optimal properties of the Bayes estimators are discussed, and as an example of the formalism, closed form expressions for the Bayes estimators for the moments of the Shannon entropy function are derived. Then numerical results are presented that compare the Bayes estimator to the frequency-counts estimator for the Shannon entropy. We also present the closed form estimators, all derived elsewhere, for the mutual information, \mathrm\ensuremathχ2 covariance, and some other statistics. (c) 1995 The American Physical Society