Analyzing Text Representations by Measuring Task Alignment

2023/05/31 by César González-Gutiérrez, Gonzalez-Gutierrez, Cesar, Audi Primadhanty +5 · 2 citations
Computer Science · #Artificial intelligence #Cluster analysis #Computation and Language (cs.CL) #Computer science #FOS: Computer and information sciences #Granularity #Key (lock) #Natural Language Processing Techniques #Natural language processing #Pattern recognition (psychology) #Representation (politics) #Space (punctuation) #Task (project management) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2305.19747

openalex publication_date 2023/05/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04

Abstract

Textual representations based on pre-trained language models are key, especially in few-shot learning scenarios. What makes a representation good for text classification? Is it due to the geometric properties of the space or because it is well aligned with the task? We hypothesize the second claim. To test it, we develop a task alignment score based on hierarchical clustering that measures alignment at different levels of granularity. Our experiments on text classification validate our hypothesis by showing that task alignment can explain the classification performance of a given representation.

Cited by

• Do Prompts Reshape Representations? An Empirical Study of Prompting Effects on Embeddings
• Domain Pre-training Impact on Representations

Related

• Analyzing Text Representations under Tight Annotation Budgets: Measuring Structural Alignment
• Task-Specific Embeddings for Ante-Hoc Explainable Text Classification
• Recurrent Alignment with Hard Attention for Hierarchical Text Rating
• Coarse-to-Fine: Hierarchical Multi-task Learning for Natural Language Understanding
• Disentangling Task Relations for Few-shot Text Classification via Self-Supervised Hierarchical Task Clustering