Nonparametric Estimation and On-Line Prediction for General Stationary Ergodic Sources

2010/02/24 by Joe Suzuki, Suzuki, Joe
Computer Science · Mathematics · #Algorithms and Data Compression #Artificial Intelligence (cs.AI) #Computability, Logic, AI Algorithms #FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Machine Learning and Algorithms #Probability (math.PR) #cs.AI #cs.IT #math.IT #math.PR

paper · pdf · doi:10.48550/arxiv.1002.4453

openalex publication_date 2010/02/24 · arxiv created 2010/06/26 · arxiv updated 2010/06/29 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/28

Abstract

We proposed a learning algorithm for nonparametric estimation and on-line prediction for general stationary ergodic sources. We prepare histograms each of which estimates the probability as a finite distribution, and mixture them with weights to construct an estimator. The whole analysis is based on measure theory. The estimator works whether the source is discrete or continuous. If it is stationary ergodic, then the measure theoretically given Kullback-Leibler information divided by the sequence length n converges to zero as n goes to infinity. In particular, for continuous sources, the method does not require existence of a probability density function.

Related

• Compression-based methods for nonparametric density estimation, on-line prediction, regression and classification for time series
• Some limits to nonparametric estimation for ergodic processes
• A consistent deterministic regression tree for non-parametric prediction of time series
• The ergodic decomposition of asymptotically mean stationary random sources
• Nonparametric statistical inference for the context tree of a stationary ergodic process