2008/11/19 by R. C. Venkatesan, Venkatesan, R. C., A. Plastino +1
Economics, Econometrics and Finance · Mathematics · Physics and Astronomy · #Complex Systems and Time Series Analysis #Data Analysis #FOS: Computer and information sciences #FOS: Physical sciences #Machine Learning (stat.ML) #Statistical Mechanics (cond-mat.stat-mech) #Statistical Mechanics and Entropy #Statistics and Probability (physics.data-an) #cond-mat.stat-mech #physics.data-an #stat.ML
paper · pdf · doi:10.48550/arxiv.0811.3174
6 pages. Expanded analysis, typographical corrections, 1 reference added
openalex publication_date 2008/11/19 · arxiv created 2009/05/01 · arxiv updated 2009/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The theoretical basis for a candidate variational principle for the information bottleneck (IB) method is formulated within the ambit of the generalized nonadditive statistics of Tsallis. Given a nonadditivity parameter q , the role of the additive duality of nonadditive statistics ( q^*=2-q ) in relating Tsallis entropies for ranges of the nonadditivity parameter q < 1 and q > 1 is described. Defining X , X , and Y to be the source alphabet, the compressed reproduction alphabet, and, the relevance variable respectively, it is demonstrated that minimization of a generalized IB (gIB) Lagrangian defined in terms of the nonadditivity parameter q^* self-consistently yields the nonadditive effective distortion measure to be the q -deformed generalized Kullback-Leibler divergence: DK-Lq[p(Y|X)||p(Y| X)] . This result is achieved without enforcing any a-priori assumptions. Next, it is proven that the q^*-deformed nonadditive free energy of the system is non-negative and convex. Finally, the update equations for the gIB method are derived. These results generalize critical features of the IB method to the case of Tsallis statistics.