Language Trees and Zipping

2001/08/31 by Dario Benedetto, Emanuele Caglioti, Vittorio Loreto · 12 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · Physics and Astronomy · #Algorithms and Data Compression #Computability, Logic, AI Algorithms #Fractal and DNA sequence analysis #cond-mat.stat-mech

paper · pdf · doi:10.1103/physrevlett.88.048702

5 pages, RevTeX4, 1 eps figure. In press in Phys. Rev. Lett. (January 2002)

arxiv created 2001/12/19 · openalex publication_date 2002/01/01 · arxiv updated 2009/11/30 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/28

Abstract

In this Letter we present a very general method for extracting information from a generic string of characters, e.g., a text, a DNA sequence, or a time series. Based on data-compression techniques, its key point is the computation of a suitable measure of the remoteness of two bodies of knowledge. We present the implementation of the method to linguistic motivated problems, featuring highly accurate results for language recognition, authorship attribution, and language classification.

Cited by

• Dictionary-based methods for information extraction
• Compression and Machine Learning: A New Perspective on Feature Space Vectors
• Sublinear Algorithms for Approximating String Compressibility
• Clustering by Compression
• The Similarity Metric
• Information Distance in Multiples
• Relative entropy via non-sequential recursive pair substitution
• From form to information: Analysing built environments in different spatial cultures
• Authorship Attribution
• A survey of modern authorship attribution methods
• Surveying Stylometry Techniques and Applications
• Equilibrium and dynamic methods when comparing an English text and its Esperanto translation