2017/05/31 by Fushing Hsieh, Hsieh Fushing, Shan-Yu Liu +2
Computer Science · Mathematics · #Artificial intelligence #Categorical variable #Computer science #Conditional entropy #Covariate #Data Visualization and Analytics #Data mining #Dependency (UML) #Entropy (arrow of time) #Machine learning #Mathematics #Mutual information #Pattern recognition (psychology) #Principle of maximum entropy #Statistical and Computational Modeling #Theoretical computer science #Time Series Analysis and Forecasting #stat.ME
paper · pdf · doi:10.1371/journal.pone.0198253
32 pages, 10 figures, 3 box pictures
arxiv created 2017/05/31 · openalex publication_date 2018/06/14 · arxiv updated 2018/07/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Data generated from a system of interest typically consists of measurements on many covariate features and possibly multiple response features across all subjects in a designated ensemble. Such data is naturally represented by one response-matrix against one covariate-matrix. A matrix lattice is an advantageous platform for simultaneously accommodating heterogeneous data types: continuous, discrete and categorical, and exploring hidden dependency among/between features and subjects. After each feature being individually renormalized with respect to its own histogram, the categorical version of mutual conditional entropy is evaluated for all pairs of response and covariate features according to the combinatorial information theory. Then, by applying Data Could Geometry (DCG) algorithmic computations on such a mutual conditional entropy matrix, multiple synergistic feature-groups are partitioned. Distinct synergistic feature-groups embrace distinct structures of dependency. The explicit details of dependency among members of synergistic features are seen through mutliscale compositions of blocks computed by a computing paradigm called Data Mechanics. We then propose a categorical pattern matching approach to establish a directed associative linkage: from the patterned response dependency to serial structured covariate dependency. The graphic display of such a directed associative linkage is termed an information flow and the degrees of association are evaluated via tree-to-tree mutual conditional entropy. This new universal way of discovering system knowledge is illustrated through five data sets. In each case, the emergent visible heterogeneity is an organization of discovered knowledge.