2021/04/15 by Xuan Wu, Wu, Xuan
Computer Science · Physics and Astronomy · #62-08 #Blind Source Separation Techniques #Computation (stat.CO) #FOS: Computer and information sciences #Scientific Research and Discoveries #Statistical Mechanics and Entropy
paper · pdf · doi:10.48550/arxiv.2104.07180
openalex publication_date 2021/04/15 · openalex created_date 2021/04/26 · openalex updated_date 2026/07/28
The Fisher information matrix provides a way to measure the amount of information given observed data based on parameters of interest. Many applications of the FIM exist in statistical modeling, system identification, and parameter estimation. We sometimes use the Monte Carlo-based method to estimate the FIM because its analytical form is often impossible or difficult to be computed in real-world models. In this paper, we review the basic method based on simultaneous perturbations and present an enhanced resampling-based method with independent simultaneous perturbations to estimate the Fisher information matrix. We conduct theoretical and numerical analysis to show its accuracy via variance reduction from O(1/N) to O(1/(nN)), where n is the sample size of the data and N is a measure of the Monte Carlo averaging. We also consider the trade-off between accuracy and computational cost.