HEFT: A Coarse-to-Fine Hierarchy for Enhancing the Efficiency and Accuracy of Language Model Reasoning

2025/09/11 by Bonnie Hill, Hill, Brennen
Computer Science · #68T05 #68T07 #68T50 #Artificial Intelligence (cs.AI) #C.4 #Computation and Language (cs.CL) #FOS: Computer and information sciences #I.2.6 #I.2.7 #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2509.09801

openalex publication_date 2025/09/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The adaptation of large language models (LLMs) to specialized reasoning tasks is fundamentally constrained by computational resources. Parameter-Efficient Fine-Tuning (PEFT) methods have emerged as a powerful solution, yet the landscape of these techniques is diverse, with distinct methods operating in either the model's weight space or its representation space. This paper investigates the hypothesis that a synergistic combination of these paradigms can unlock superior performance and efficiency. We introduce HEFT (Hierarchical Efficient Fine-Tuning), a novel hierarchical adaptation strategy that composes two distinct PEFT methods in a coarse-to-fine manner: first, a broad, foundational adaptation in the weight space using Low-Rank Adaptation (LoRA), followed by a precise, surgical refinement of internal activations using Representation Fine-Tuning (ReFT). We evaluate this approach by fine-tuning a Llama-2-7B model on the BoolQ benchmark, a challenging dataset for inferential reasoning. Our results reveal a profound synergistic effect. A model fine-tuned for only three epochs with our HEFT strategy achieves an accuracy of 85.17%, exceeding the performance of models trained for 20 epochs with either LoRA-only (85.05%) or ReFT-only (83.36%) methodologies. This work demonstrates that the thoughtful composition of PEFT methods is a potent algorithmic innovation, offering a more efficient and effective path toward advancing the reasoning capabilities of language models. By achieving superior results with a fraction of the computational budget, our findings present a principled approach to overcoming the obstacles inherent in adapting large-scale models for complex cognitive tasks.

Citations

• FedReFT: Federated Representation Fine-Tuning with All-But-Me Aggregation
• A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning
• Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
• LoRA vs Full Fine-tuning: An Illusion of Equivalence
• ReFT: Representation Finetuning for Language Models
• Does Combining Parameter-efficient Modules Improve Few-shot Transfer Accuracy?
• LoRA+: Efficient Low Rank Adaptation of Large Models
• The Falcon Series of Open Language Models
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
• LLaMA: Open and Efficient Foundation Language Models
• Towards Reasoning in Large Language Models: A Survey
• On the Effectiveness of Parameter-Efficient Fine-Tuning
• Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
• BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
• LoRA: Low-Rank Adaptation of Large Language Models
• SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
• BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
• Parameter-Efficient Transfer Learning for NLP

Related

• SIBO: A Simple Booster for Parameter-Efficient Fine-Tuning
• Specializing Smaller Language Models towards Multi-Step Reasoning
• KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models
• ReFT: Representation Finetuning for Language Models
• How Much is Too Much? Exploring LoRA Rank Trade-offs for Retaining Knowledge and Domain Robustness