LiRA: Linguistic Robust Anchoring for Cross-lingual Large Language Models

2025/10/16 by Li, Haolin, Zhang, Haipeng, Li, Mang +4
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2510.14466

Abstract

As large language models (LLMs) rapidly advance, performance on high-resource languages (e.g., English, Chinese) is nearing saturation, yet remains substantially lower for low-resource languages (e.g., Urdu, Thai) due to limited training data, machine-translation noise, and unstable cross-lingual alignment. We introduce LiRA (Linguistic Robust Anchoring for Large Language Models), a training framework that robustly improves cross-lingual representations under low-resource conditions while jointly strengthening retrieval and reasoning. LiRA comprises two modules: (i) Arca (Anchored Representation Composition Architecture), which anchors low-resource languages to an English semantic space via anchor-based alignment and multi-agent collaborative encoding, preserving geometric stability in a shared embedding space; and (ii) LaSR (Language-coupled Semantic Reasoner), which adds a language-aware lightweight reasoning head with consistency regularization on top of Arca's multilingual representations, unifying the training objective to enhance cross-lingual understanding, retrieval, and reasoning robustness. We further construct and release a multilingual product retrieval dataset covering five Southeast Asian and two South Asian languages. Experiments across low-resource benchmarks (cross-lingual retrieval, semantic similarity, and reasoning) show consistent gains and robustness under few-shot and noise-amplified settings; ablations validate the contribution of both Arca and LaSR. Code will be released on GitHub and the dataset on Hugging Face.

Citations

• Improving Multilingual Language Models by Aligning Representations through Steering
• XRAG: Cross-lingual Retrieval-Augmented Generation
• MMTEB: Massive Multilingual Text Embedding Benchmark
• A Survey of Multilingual Reasoning in Language Models
• A Three-Pronged Approach to Cross-Lingual Adaptation with Multilingual LLMs
• Data Contamination Can Cross Language Barriers
• Adaptive Cross-lingual Text Classification through In-Context One-Shot Demonstrations
• Cross-lingual Contextualized Phrase Retrieval
• How do Large Language Models Handle Multilingualism?
• Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval
• Revisiting Machine Translation for Cross-lingual Classification
• xPQA: Cross-Lingual Product Question Answering across 12 Languages
• Survey of Low-Resource Machine Translation
• Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning
• XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
• Unsupervised Cross-lingual Representation Learning at Scale
• Semantic Drift in Multilingual Representations
• Hilbert space embeddings and metrics on probability measures
• Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Models

Related

• Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineering
• Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging -- An Open Recipe
• LLM-powered Data Augmentation for Enhanced Cross-lingual Performance
• Extrapolating Large Language Models to Non-English by Aligning Languages
• Uncovering Cross-Linguistic Disparities in LLMs using Sparse Autoencoders