2020/01/01 by Kan Li, José C. Prı́ncipe, Li, Kan +2
Computer Science · Engineering · Mathematics · #Advanced Adaptive Filtering Techniques #Blind Source Separation Techniques #FOS: Computer and information sciences #Image and Signal Denoising Methods #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #cs.IT #cs.LG #math.IT #stat.ML
paper · pdf · doi:10.48550/arxiv.2001.00265
10 pages, 3 figures, 2 tables. arXiv admin note: text overlap with arXiv:1912.04530
arxiv created 2020/01/01 · openalex publication_date 2020/01/01 · arxiv updated 2020/01/03 · openalex created_date 2020/01/10 · openalex updated_date 2026/07/28
Kernel methods form a theoretically-grounded, powerful and versatile framework to solve nonlinear problems in signal processing and machine learning. The standard approach relies on the kernel trick to perform pairwise evaluations of a kernel function, leading to scalability issues for large datasets due to its linear and superlinear growth with respect to the training data. Recently, we proposed no-trick (NT) kernel adaptive filtering (KAF) that leverages explicit feature space mappings using data-independent basis with constant complexity. The inner product defined by the feature mapping corresponds to a positive-definite finite-rank kernel that induces a finite-dimensional reproducing kernel Hilbert space (RKHS). Information theoretic learning (ITL) is a framework where information theory descriptors based on non-parametric estimator of Renyi entropy replace conventional second-order statistics for the design of adaptive systems. An RKHS for ITL defined on a space of probability density functions simplifies statistical inference for supervised or unsupervised learning. ITL criteria take into account the higher-order statistical behavior of the systems and signals as desired. However, this comes at a cost of increased computational complexity. In this paper, we extend the NT kernel concept to ITL for improved information extraction from the signal without compromising scalability. Specifically, we focus on a family of fast, scalable, and accurate estimators for ITL using explicit inner product space (EIPS) kernels. We demonstrate the superior performance of EIPS-ITL estimators and combined NT-KAF using EIPS-ITL cost functions through experiments.