Data-driven Methods of Extracting Text Structure and Information Transfer

2025/09/16 by Shinichi Honna, Honna, Shinichi, Taichi Murayama +3
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2509.12999

openalex publication_date 2025/09/16 · openalex created_date 2025/10/18 · openalex updated_date 2026/07/28

Abstract

The Anna Karenina Principle (AKP) holds that success requires satisfying a small set of essential conditions, whereas failure takes diverse forms. We test AKP, its reverse, and two further patterns described as ordered and noisy across novels, online encyclopedias, research papers, and movies. Texts are represented as sequences of functional blocks, and convergence is assessed in transition order and position. Results show that structural principles vary by medium: novels follow reverse AKP in order, Wikipedia combines AKP with ordered patterns, academic papers display reverse AKP in order but remain noisy in position, and movies diverge by genre. Success therefore depends on structural constraints that are specific to each medium, while failure assumes different shapes across domains.

Citations

• DiscoScore: Evaluating Text Generation with BERT and Discourse Coherence
• Adversarial Multi-task Learning for Text Classification
• The emotional arcs of stories are dominated by six basic shapes
• The Bayesian Block Algorithm
• The Anna Karenina principle: A concept for the explanation of success in science
• Modeling Local Coherence: An Entity-Based Approach

Related

• Beyond Zipf's law: Modeling the structure of human language
• Robustness of sentence length measures in written texts
• A quantitative analysis of concepts and semantic structure in written language: Long range correlations in dynamics of texts
• Dataset for paper "Sameness entices, but novelty enchants in fanfiction online"
• The Problem of Alignment