BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions

2019/05/24 by Christopher Clark, Kenton Lee, Clark, Christopher +11 · 541 citations
Computer Science · Psychology · #Archaeology #Computation and Language (cs.CL) #Epistemology #FOS: Computer and information sciences #History #Natural (archaeology) #Natural Language Processing Techniques #Philosophy #Psychology #Semantic Web and Ontologies #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.1905.10044

published in arXiv (Cornell University) (Cornell University) · In NAACL 2019

arxiv created 2019/05/24 · openalex publication_date 2019/05/24 · arxiv updated 2019/05/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper we study yes/no questions that are naturally occurring --- meaning that they are generated in unprompted and unconstrained settings. We build a reading comprehension dataset, BoolQ, of such questions, and show that they are unexpectedly challenging. They often query for complex, non-factoid information, and require difficult entailment-like inference to solve. We also explore the effectiveness of a range of transfer learning baselines. We find that transferring from entailment data is more effective than transferring from paraphrase or extractive QA data, and that it, surprisingly, continues to be very beneficial even when starting from massive pre-trained language models such as BERT. Our best method trains BERT on MultiNLI and then re-trains it on our train set. It achieves 80.4% accuracy compared to 90% accuracy of human annotators (and 62% majority-baseline), leaving a significant gap for future work.

Citations

• Bidirectional Attention Flow for Machine Comprehension
• Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
• VQA: Visual Question Answering

Cited by

• Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Loss
• Single LLM Debate, MoLaCE: Mixture of Latent Concept Experts Against Confirmation Bias
• Interpretable Safety Alignment via SAE-Constructed Low-Rank Subspace Adaptation
• The Quest for Winning Tickets in Low-Rank Adapters
• AFA-LoRA: Enabling Non-Linear Adaptations in LoRA with Activation Function Annealing
• LLMBoost: Make Large Language Models Stronger with Boosting
• Conformal Cascade: Distribution-Free Accuracy Guarantees for Multi-Tier LLM Inference
• Deep Delta Learning
• Broken Words, Broken Performance: Effect of Tokenization on Performance of LLMs
• Rethinking Output Alignment For 1-bit Post-Training Quantization of Large Language Models
• Memory-Efficient Acceleration of Block Low-Rank Foundation Models on Resource Constrained GPUs
• LLM-CAS: Dynamic Neuron Perturbation for Real-Time Hallucination Correction
• Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layers
• Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensation
• T5Gemma 2: Seeing, Reading, and Understanding Longer
• VersatileFFN: Achieving Parameter Efficiency in LLMs via Adaptive Wide-and-Deep Reuse
• SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations
• DTop-p MoE: Sparsity-Controlled Dynamic Top-p MoE for Foundation Model Pre-training
• Towards Effective Model Editing for LLM Personalization
• MIDUS: Memory-Infused Depth Up-Scaling
• Resting Neurons, Active Insights: Improving Input Sparsification for Large Language Models
• Exact Flow Linear Attention: Exact Solution from Continuous-Time Dynamics
• Low-Rank Compression of Language Models via Differentiable Rank Selection
• Scaling Behavior of Discrete Diffusion Language Models
• GatedFWA: Linear Flash Windowed Attention with Gated Associative Memory
• PCMind-2.1-Kaiyuan-2B Technical Report
• LIME: Making LLM Data More Efficient with Linguistic Metadata Embeddings
• Leveraging KV Similarity for Online Structured Pruning in LLMs
• Greedy Alignment Principle for Optimizer Selection
• SignRoundV2: Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMs
• Context-Aware Mixture-of-Experts Inference on CXL-Enabled GPU-NDP Systems
• PEFT-Factory: Unified Parameter-Efficient Fine-Tuning of Autoregressive Large Language Models
• Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling
• HBLLM: Wavelet-Enhanced High-Fidelity 1-Bit Quantization for LLMs
• Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relations
• Dripper: Token-Efficient Main HTML Extraction with a Lightweight LM
• PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark
• CafeQ: Calibration-free Quantization via Learned Transformations and Adaptive Rounding
• ROOT: Robust Orthogonalized Optimizer for Neural Network Training
• Mosaic Pruning: A Hierarchical Framework for Generalizable Pruning of Mixture-of-Experts Models
• R2Q: Towards Robust 2-Bit Large Language Models via Residual Refinement Quantization
• E³-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models
• AICC: Parse HTML Finer, Make Models Better -- A 7.3T AI-Ready Corpus Built by a Model-Based HTML Parser
• Automatic Pruning Discovery for Large Language Models
• Dynamic Nested Hierarchies: Pioneering Self-Evolution in Machine Learning Architectures for Lifelong Intelligence
• OTARo: Once Tuning for All Precisions toward Robust On-Device LLMs
• GateRA: Token-Aware Modulation for Parameter-Efficient Fine-Tuning
• A Multifaceted Analysis of Negative Bias in Large Language Models through the Lens of Parametric Knowledge
• On-Device Fine-Tuning via Backprop-Free Zeroth-Order Optimization
• ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference
• Safety-Preserving PTQ via Contrastive Alignment Loss
• Range Asymmetric Numeral Systems-Based Lightweight Intermediate Feature Compression for Split Computing of Deep Neural Networks
• SpecQuant: Spectral Decomposition and Adaptive Truncation for Ultra-Low-Bit LLMs Quantization
• Testing Question Answering Software with Context-Driven Question Generation
• Routing Manifold Alignment Improves Generalization of Mixture-of-Experts LLMs
• Learning to Focus: Focal Attention for Selective and Scalable Transformers
• MobileLLM-Pro Technical Report
• MuonAll: Muon Variant for Efficient Finetuning of Large Language Models
• DyKAF: Dynamical Kronecker Approximation of the Fisher Information Matrix for Gradient Preconditioning
• Motif 2 12.7B technical report
• PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
• Memory- and Latency-Constrained Inference of Large Language Models via Adaptive Split Computing
• CryptoMoE: Privacy-Preserving and Scalable Mixture of Experts Inference via Balanced Expert Routing
• ConMeZO: Adaptive Descent-Direction Sampling for Gradient-Free Finetuning of Large Language Models
• Optimizing Native Sparse Attention with Latent Attention and Local Global Alternating Strategies
• A Hierarchical Imprecise Probability Approach to Reliability Assessment of Large Language Models
• TetraJet-v2: Accurate NVFP4 Training for Large Language Models with Oscillation Suppression and Outlier Control
• A Comparative Analysis of LLM Adaptation: SFT, LoRA, and ICL in Data-Scarce Scenarios
• Encoder-Decoder or Decoder-Only? Revisiting Encoder-Decoder Large Language Model
• 1+1>2: A Synergistic Sparse and Low-Rank Compression Method for Large Language Models
• MossNet: Mixture of State-Space Experts is a Multi-Head Attention
• Gaperon: A Peppered English-French Generative Language Model Suite
• Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges
• Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers
• Mixture-of-Depths Attention
• Massive Spikes in LLMs are Bias Vectors: Mechanistic Uncovering and Spike-Free Quantization
• Robustness as an Emergent Property of Task Performance
• Can LLMs Estimate Cognitive Complexity of Reading Comprehension Items?
• MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling
• LoRA-DA: Data-Aware Initialization for Low-Rank Adaptation via Asymptotic Analysis
• What Limits Agentic Systems Efficiency?
• From Cross-Task Examples to In-Task Prompts: A Graph-Based Pseudo-Labeling Framework for In-context Learning
• Charting the European LLM Benchmarking Landscape: A New Taxonomy and a Set of Best Practices
• Calibrating and Rotating: A Unified Framework for Weight Conditioning in PEFT
• FALQON: Accelerating LoRA Fine-tuning with Low-Bit Floating-Point Arithmetic
• MeCeFO: Enhancing LLM Training Robustness via Fault-Tolerant Optimization
• ChessQA: Evaluating Large Language Models for Chess Understanding
• ScaLoRA: Optimally Scaled Low-Rank Adaptation for Efficient High-Rank Fine-Tuning
• Multi-Agent Evolve: LLM Self-Improve through Co-evolution
• Simple Denoising Diffusion Language Models
• SeeDNorm: Self-Rescaled Dynamic Normalization
• MMPersuade: A Dataset and Evaluation Framework for Multimodal Persuasion
• TELL-TALE: Task Efficient LLMs with Task Aware Layer Elimination
• Low-Resource Dialect Adaptation of Large Language Models: A French Dialect Case-Study
• Label Smoothing Improves Gradient Ascent in LLM Unlearning
• No Mean Feat: Simple, Strong Baselines for Context Compression
• Systematic Evaluation of Uncertainty Estimation Methods in Large Language Models
• Citation Failure: Definition, Analysis and Efficient Mitigation
• GaLLoP: Gradient-based Sparse Learning on Low-Magnitude Parameters
• Zhyper: Factorized Hypernetworks for Conditioned LLM Fine-Tuning
• Energy-Efficient and Dequantization-Free Q-LLMs: A Spiking Neural Network Approach to Salient Value Mitigation
• Restoring Pruned Large Language Models via Lost Component Compensation
• Binary Quadratic Quantization: Beyond First-Order Quantization for Real-Valued Matrix Compression
• NeuroAda: Activating Each Neuron's Potential for Parameter-Efficient Fine-Tuning
• ScaleNet: Scaling up Pretrained Neural Networks with Incremental Parameters
• From Local to Global: Revisiting Structured Pruning Paradigms for Large Language Models
• ReXMoE: Reusing Experts with Minimal Overhead in Mixture-of-Experts
• Vocab Diet: Reshaping the Vocabulary of LLMs with Vector Arithmetic
• Online Mixture of Experts: No-Regret Learning for Optimal Collective Decision-Making
• DistilLock: Safeguarding LLMs from Unauthorized Knowledge Distillation on the Edge
• FraQAT: Quantization Aware Training with Fractional bits
• First Attentions Last: Better Exploiting First Attentions for Efficient Transformer Training
• Towards Reversible Model Merging For Low-rank Weights
• REAP the Experts: Why Pruning Prevails for One-Shot MoE compression
• End-to-End Multi-Modal Diffusion Mamba
• Tahakom LLM Guidelines and Recipes: From Pre-training Data to an Arabic LLM
• PIShield: Detecting Prompt Injection Attacks via Intrinsic LLM Features
• OPLoRA: Orthogonal Projection LoRA Prevents Catastrophic Forgetting during Parameter-Efficient Fine-Tuning
• Deconstructing Attention: Investigating Design Principles for Effective Language Modeling
• Neural Weight Compression for Language Models
• ShishuLM: Lightweight Language Model with Hybrid Decoder-MLP Architecture and Paired Weight Sharing
• RePro: Training Language Models to Faithfully Recycle the Web for Pretraining
• Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization
• Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods
• CTR-LoRA: Curvature-Aware and Trust-Region Guided Low-Rank Adaptation for Large Language Models
• Tapered Language Models
• Decoupled DiLoCo for Resilient Distributed Pre-training
• Efficient Resource-Constrained Training of Transformers via Subspace Optimization
• KORMo: Korean Open Reasoning Model for Everyone
• AILoRA: Function-Aware Asymmetric Initialization for Low-Rank Adaptation of Large Language Models
• RCPU: Rotation-Constrained Error Compensation for Structured Pruning of Large Language Models
• Beyond Sunk Costs: Boosting LLM Pre-training Efficiency via Orthogonal Growth of Mixture-of-Experts
• Single layer tiny Co⁴ outpaces GPT-2 and GPT-BERT
• SliceFine: The Universal Winning-Slice Hypothesis for Pretrained Networks
• Next Semantic Scale Prediction via Hierarchical Diffusion Language Models
• When Benchmarks Age: Temporal Misalignment through Large Language Model Factuality Evaluation
• Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts
• POME: Post Optimization Model Edit via Muon-style Projection
• Mixture of Neuron Experts
• Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM
• AMAQ: Adaptive Mixed-bit Activation Quantization for Collaborative Parameter Efficient Fine-tuning
• BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining
• GraphGhost: Tracing Structures Behind Large Language Models
• Boomerang Distillation Enables Zero-Shot Model Size Interpolation
• Recover-LoRA: Data-Free Accuracy Recovery of Degraded Language Models via Low-Rank Adaptation
• SpikingMamba: Towards Energy-Efficient Large Language Models via Knowledge Distillation from Mamba
• COLE: a Comprehensive Benchmark for French Language Understanding Evaluation
• Rounding-Guided Backdoor Injection in Deep Learning Model Quantization
• The Unseen Frontier: Pushing the Limits of LLM Sparsity with Surrogate-Free ADMM
• Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner
• Energy-Regularized Sequential Model Editing on Hyperspheres
• ElasWave: An Elastic-Native System for Scalable Hybrid-Parallel Training
• Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs
• BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responses
• Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training
• Training Matryoshka Mixture-of-Experts for Elastic Inference-Time Expert Utilization
• CAST: Continuous and Differentiable Semi-Structured Sparsity-Aware Training for Large Language Models
• Understanding the Mixture-of-Experts with Nadaraya-Watson Kernel
• The Flaw of Averages: Quantifying Uniformity of Performance on Benchmarks
• MixtureVitae: Open Web-Scale Pretraining Dataset With High Quality Instruction and Reasoning Data Built from Permissive-First Text Sources
• Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs
• Uni-X: Mitigating Modality Conflict with a Two-End-Separated Architecture for Unified Multimodal Models
• Conda: Column-Normalized Adam for Training Large Language Models Faster
• Negative Pre-activations Differentiate Syntax
• CURA: Size Isnt All You Need -- A Compact Universal Architecture for On-Device Intelligence
• Pretraining with hierarchical memories: separating long-tail and common knowledge
• Train Once, Answer All: Many Pretraining Experiments for the Cost of One
• SDQ-LLM: Sigma-Delta Quantization for 1-bit LLMs of any size
• Effective Quantization of Muon Optimizer States
• PT²-LLM: Post-Training Ternarization for Large Language Models
• MoE-PHDS: One MoE checkpoint for flexible runtime sparsity
• Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
• JGU Mainz's Submission to the WMT25 Shared Task on LLMs with Limited Resources for Slavic Languages: MT and QA
• Lightweight error mitigation strategies for post-training N:M activation sparsity in LLMs
• Rethinking RoPE Scaling in Quantized LLM: Theory, Outlier, and Channel-Band Analysis with Weight Rescaling
• How Accurate Are LLMs at Multi-Question Answering on Conversational Transcripts?
• Induction Signatures Are Not Enough: A Matched-Compute Study of Load-Bearing Structure in In-Context Learning
• Blockwise Hadamard high-Rank Adaptation for Parameter-Efficient LLM Fine-Tuning
• Dual-Head Reasoning Distillation: Improving Classifier Accuracy with Train-Time-Only Reasoning
• CLUE: Conflict-guided Localization for LLM Unlearning Framework
• Performance Consistency of Learning Methods for Information Retrieval Tasks
• Thinking Augmented Pre-training
• Let's Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports
• Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editing
• GyRot: Leveraging Hidden Synergy Between Rotation and Fine-Grained Group Quantization for Low-Bit LLM Inference
• WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
• Learning to Select, Not Relearn: Hard-Routed Mixtures of Reasoning LoRAs
• EMO: Pretraining Mixture of Experts for Emergent Modularity
• GradMAP: Faster Layer Pruning with Gradient Metric and Projection Compensation
• HyperAdapt: Simple High-Rank Adaptation
• TiKMiX: Take Data Influence into Dynamic Mixture for Language Model Pre-training
• On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs
• TASO: Task-Aligned Sparse Optimization for Parameter-Efficient Model Adaptation
• QWHA: Quantization-Aware Walsh-Hadamard Adaptation for Parameter-Efficient Fine-Tuning on Large Language Models
• PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
• DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruning
• Distribution-Aligned Decoding for Efficient LLM Task Adaptation
• What's Not on the Plate? Rethinking Food Computing through Indigenous Indian Datasets
• Once Upon a Time: Interactive Learning for Storytelling with Small Language Models
• FURINA: Free from Unmergeable Router via LINear Aggregation of mixed experts
• MoE-Inference-Bench: Performance Evaluation of Mixture of Expert Large Language and Vision Models
• Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generation
• CBP-Tuning: Efficient Local Customization for Black-box Large Language Models
• AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Models
• Optimal Brain Restoration for Joint Quantization and Sparsification of LLMs
• GAMA: A General Anonymizing Multi-Agent System for Privacy Preservation Enhanced by Domain Rules and Disproof Mechanism
• Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMs
• HEFT: A Coarse-to-Fine Hierarchy for Enhancing the Efficiency and Accuracy of Language Model Reasoning
• Building High-Quality Datasets for Portuguese LLMs: From Common Crawl Snapshots to Industrial-Grade Corpora
• Explainable Semantic Text Relations: A Question-Answering Framework for Comparing Document Content
• Open-sci-ref-0.01: open and reproducible reference baselines for language model and dataset comparison
• Interpreting the Effects of Quantization on LLMs
• Causal Attention with Lookahead Keys
• CTCC: A Robust and Stealthy Fingerprinting Framework for Large Language Models via Cross-Turn Contextual Correlation Backdoor
• On Robustness and Reliability of Benchmark-Based Evaluation of LLMs
• A Probabilistic Inference Scaling Theory for LLM Self-Correction
• TeRA: Vector-based Random Tensor Network for High-Rank Adaptation of Large Language Models
• From Injection to Defense: Constructing Edit-Based Fingerprints for Large Language Models
• Calibration and Transformation-Free Weight-Only LLMs Quantization via Dynamic Grouping
• Language Models are Few-Shot Learners
• EverTracer: Hunting Stolen Large Language Models via Stealthy and Robust Probabilistic Fingerprint
• Implicit Reasoning in Large Language Models: A Comprehensive Survey
• Towards Fundamental Language Models: Does Linguistic Competence Scale with Model Size?
• LExI: Layer-Adaptive Active Experts for Efficient MoE Model Inference
• GEM: A Scale-Aware and Distribution-Sensitive Sparse Fine-Tuning Framework for Effective Downstream Adaptation
• GradES: Significantly Faster Training in Transformers with Gradient-Based Early Stopping
• Natural Context Drift Undermines the Natural Language Understanding of Large Language Models
• REFRAG: Rethinking RAG based Decoding
• QueryBandits for Hallucination Mitigation: Exploiting Semantic Features for No-Regret Rewriting
• MEPT: Mixture of Expert Prompt Tuning as a Manifold Mapper
• Unlocking the Effectiveness of LoRA-FP for Seamless Transfer Implantation of Fingerprints in Downstream Models
• Router Upcycling: Leveraging Mixture-of-Routers in Mixture-of-Experts Upcycling
• DTRNet: Dynamic Token Routing Network to Reduce Quadratic Costs in Transformers
• Universal Properties of Activation Sparsity in Modern Large Language Models
• Metis: Training LLMs with FP4 Quantization
• PDTrim: Targeted Pruning for Prefill-Decode Disaggregation in Inference
• ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning
• Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection
• FedReFT: Federated Representation Fine-Tuning with All-But-Me Aggregation
• QuesGenie: Intelligent Multimodal Question Generation
• SLM-Bench: A Comprehensive Benchmark of Small Language Models on Environmental Impacts--Extended Version
• Beyond Benchmark: LLMs Evaluation with an Anthropomorphic and Value-oriented Roadmap
• Integral Transformer: Denoising Attention, Not Too Much Not Too Little
• DualSparse-MoE: Coordinating Tensor/Neuron-Level Sparsity with Expert Partition and Reconstruction
• WISCA: A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling
• Identifying and Answering Questions with False Assumptions: An Interpretable Approach
• Evaluating and Characterizing Human Rationales
• Sycophancy under Pressure: Evaluating and Mitigating Sycophantic Bias via Adversarial Dialogues in Scientific QA
• GLASS: Global-Local Aggregation for Inference-time Sparsification of LLMs
• Z-Pruner: Post-Training Pruning of Large Language Models for Efficiency without Retraining
• Maximum Score Routing For Mixture-of-Experts
• KLUE: Korean Language Understanding Evaluation
• FedSODA: Federated Fine-tuning of LLMs via Similarity Group Pruning and Orchestrated Distillation Alignment
• Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation
• LLM Compression: How Far Can We Go in Balancing Size and Performance?
• MoNaCo: More Natural and Complex Questions for Reasoning Across Dozens of Documents
• Copyright Protection for Large Language Models: A Survey of Methods, Challenges, and Trends
• Shatter: An Efficient Transformer Encoder with Single-Headed Self-Attention and Relative Sequence Partitioning
• BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining
• Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings
• Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regime
• Align, Don't Divide: Revisiting the LoRA Architecture in Multi-Task Learning
• LLM Data Selection and Utilization via Dynamic Bi-level Optimization
• FlexQ: Efficient Post-training INT6 Quantization for LLM Serving via Algorithm-System Co-Design
• Forgetting: A New Mechanism Towards Better Large Language Model Fine-tuning
• Tensorized Clustered LoRA Merging for Multi-Task Interference
• Exploring Layer-wise Information Effectiveness for Post-Training Quantization in Small Language Models
• CAMERA: Multi-Matrix Joint Compression for MoE Models via Micro-Expert Redundancy Analysis
• Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models
• Kron-LoRA: Hybrid Kronecker-LoRA Adapters for Scalable, Sustainable Fine-tuning
• FPEdit: Robust LLM Fingerprinting through Localized Parameter Editing
• Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
• SpeechR: A Benchmark for Speech Reasoning in Large Audio-Language Models
• EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
• Towards Higher Effective Rank in Parameter-efficient Fine-tuning using Khatri--Rao Product
• Out-of-Context Abduction: LLMs Make Inferences About Procedural Data Leveraging Declarative Facts in Earlier Training Data
• EMA Without the Lag: Bias-Corrected Iterate Averaging Schemes
• Unveiling Super Experts in Mixture-of-Experts Large Language Models
• LENS: Learning Ensemble Confidence from Neural States for Multi-LLM Answer Integration
• Is Large Language Model Performance on Reasoning Tasks Impacted by Different Ways Questions Are Asked?
• OASIS: Outlier-Aware LUT-Based GEMM with Dual-Side Quantization for LLM Inference Acceleration
• From Benchmarks to Skills: Low-Rank Factors for LLM Evaluation
• Diffusion Beats Autoregressive in Data-Constrained Settings
• Basic Reading Distillation
• DeltaLLM: A Training-Free Framework Exploiting Temporal Sparsity for Efficient Edge LLM Inference
• Diverse LLMs or Diverse Question Interpretations? That is the Ensembling Question
• Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method
• Prune&Comp: Free Lunch for Layer-Pruned LLMs via Iterative Pruning with Magnitude Compensation
• Exploring the Impact of Instruction-Tuning on LLM's Susceptibility to Misinformation
• UniPool: A Globally Shared Expert Pool for Mixture-of-Experts
• Shaping capabilities with token-level data filtering
• MultiModalQA: Complex Question Answering over Text, Tables and Images
• A Comprehensive Evaluation on Quantization Techniques for Large Language Models
• SimLens for Early Exit in Large Language Models: Eliciting Accurate Latent Predictions with One More Token
• SpeLLM: Character-Level Multi-Head Decoding
• Enhancing Cross-task Transfer of Large Language Models via Activation Steering
• Language Models Improve When Pretraining Data Matches Target Tasks
• DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts
• LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters
• Contextual feature extraction hierarchies converge in large language models and the brain
• Olica: Efficient Structured Pruning of Large Language Models without Retraining
• OAT-Rephrase: Optimization-Aware Training Data Rephrasing for Zeroth-Order LLM Fine-Tuning
• SDMPrune: Self-Distillation MLP Pruning for Efficient Large Language Models
• First-Order Error Matters: Accurate Compensation for Quantized Large Language Models
• GeLaCo: An Evolutionary Approach to Layer Compression
• Deep Hidden Cognition Facilitates Reliable Chain-of-Thought Reasoning
• FusionFactory: Fusing LLM Capabilities with Multi-LLM Log Data
• DATE-LM: Benchmarking Data Attribution Evaluation for Large Language Models
• CompassJudger-2: Towards Generalist Judge Model via Verifiable Rewards
• Detecting and Pruning Prominent but Detrimental Neurons in Large Language Models
• BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity
• Invariant-based Robust Weights Watermark for Large Language Models
• From Ambiguity to Accuracy: The Transformative Effect of Coreference Resolution on Retrieval-Augmented Generation systems
• SAS: Simulated Attention Score
• COALA: Numerically Stable and Efficient Framework for Context-Aware Low-Rank Approximation
• Stable Preference Optimization: A Bilevel Approach to Catastrophic Preference Shift
• FlexOlmo: Open Language Models for Flexible Data Use
• On the Effect of Uncertainty on Layer-wise Inference Dynamics
• Aligned Textual Scoring Rules
• Tokenizer-Agnostic Engram Module
• Steering Information Utility in Key-Value Memory for Language Model Post-Training
• Train-before-Test Harmonizes Language Model Rankings
• LoSiA: Efficient High-Rank Fine-Tuning via Subnet Localization and Optimization
• GradOT: Training-free Gradient-preserving Offsite-tuning for Large Language Models
• MGAA: Multi-Granular Adaptive Allocation fof Low-Rank Compression of LLMs
• DistZO2: High-Throughput and Memory-Efficient Zeroth-Order Fine-tuning LLMs with Distributed Parallel Computing
• High-Layer Attention Pruning with Rescaling
• MuRating: A High Quality Data Selecting Approach to Multilingual Large Language Model Pretraining
• Curvature-Weighted Capacity Allocation: A Minimum Description Length Framework for Layer-Adaptive Large Language Model Optimization
• LEDOM: Reverse Language Model
• Eka-Eval: An Evaluation Framework for Low-Resource Multilingual Large Language Models
• La RoSA: Enhancing LLM Efficiency via Layerwise Rotated Sparse Activation
• MoNE: Replacing Redundant Experts with Lightweight Novices for Structured Pruning of MoE
• Scaling Laws Are Unreliable for Downstream Tasks: A Reality Check
• Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environments
• Spectra 1.1: Scaling Laws and Efficient Inference for Ternary Language Models
• Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
• Towards Distributed Neural Architectures
• Memory Savings at What Cost? A Study of Alternatives to Backpropagation
• AutoMixer: Checkpoint Artifacts as Automatic Data Mixers
• Data Efficacy for Language Model Training
• Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inference
• Model Editing as a Double-Edged Sword: Steering Agent Ethical Behavior Toward Beneficence or Harm
• GPTailor: Large Language Model Pruning Through Layer Cutting and Stitching
• DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMs
• Adapt Once, Thrive with Updates: Transferable Parameter-Efficient Fine-Tuning on Evolving Base Models
• Tensor-Parallelism with Partially Synchronized Activations
• Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations
• EQuARX: Efficient Quantized AllReduce in XLA for Distributed Machine Learning Acceleration
• A geometric framework for momentum-based optimizers for low-rank training
• Revisiting LoRA through the Lens of Parameter Redundancy: Spectral Encoding Helps
• Efficient and Privacy-Preserving Soft Prompt Transfer for LLMs
• Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights
• SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsity
• NeurIPS 2025 E2LM Competition : Early Training Evaluation of Language Models
• Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality
• MoORE: SVD-based Model MoE-ization for Conflict- and Oblivion-Resistant Multi-Task Adaptation
• Improving LoRA with Variational Learning
• DualEdit: Mitigating Safety Fallback in LLM Backdoor Editing via Affirmation-Refusal Regulation
• Just Go Parallel: Improving the Multilingual Capabilities of Large Language Models
• TensorSLM: Energy-efficient Embedding Compression of Sub-billion Parameter Language Models on Low-end Devices
• EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
• GTA: Grouped-head latenT Attention
• MEraser: An Effective Fingerprint Erasure Approach for Large Language Models
• Mixture-of-Experts Can Surpass Dense LLMs Under Strictly Equal Resource
• LoRA-Gen: Specializing Large Language Model via Online LoRA Generation
• Promoting Ensemble Diversity with Interactive Bayesian Distributional Robustness for Fine-tuning Foundation Models
• One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers
• Beyond Random Sampling: Efficient Language Model Pretraining via Curriculum Learning
• Towards Open Foundation Language Model and Corpus for Macedonian: A Low-Resource Language
• Learning Distribution-Wise Control in Representation Space for Language Models
• MoA: Heterogeneous Mixture of Adapters for Parameter-Efficient Fine-Tuning of Large Language Models
• Come Together, But Not Right Now: A Progressive Strategy to Boost Low-Rank Adaptation
• DynamicMind: A Tri-Mode Thinking System for Large Language Models
• Eigenspectrum Analysis of Neural Networks without Aspect Ratio Bias
• Beyond Low-rank Decomposition: A Shortcut Approach for Efficient On-Device Learning
• MesaNet: Sequence Modeling by Locally Optimal Test-Time Training
• From Understanding to Generation: An Efficient Shortcut for Evaluating Language Models
• Accurate Sublayer Pruning for Large Language Models by Exploiting Latency and Tunability Information
• APT: Improving Specialist LLM Performance with Weakness Case Acquisition and Iterative Preference Training
• TokAlign: Efficient Vocabulary Adaptation via Token Alignment
• SkipGPT: Dynamic Layer Pruning Reinvented with Token Awareness and Module Decoupling
• Faster MoE LLM Inference for Extremely Large Models
• PoLAR: Polar-Decomposed Low-Rank Adapter Representation
• CLONE: Customizing LLMs for Efficient Latency-Aware Inference at the Edge
• Adaptive Task Vectors for Large Language Models
• DiaBlo: Diagonal Blocks Are Sufficient For Finetuning
• ESGenius: Benchmarking LLMs on Environmental, Social, and Governance (ESG) and Sustainability Knowledge
• Taming LLMs by Scaling Learning Rates with Gradient Grouping
• FlowNIB: An Information Bottleneck Analysis of Bidirectional vs. Unidirectional Language Models
• Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks
• BadLingual: A Novel Lingual-Backdoor Attack against Large Language Models
• A Simple Linear Patch Revives Layer-Pruned Large Language Models
• Stepsize anything: A unified learning rate schedule for budgeted-iteration training
• SUMO: Subspace-Aware Moment-Orthogonalization for Accelerating Memory-Efficient LLM Training
• Revisiting Epistemic Markers in Confidence Estimation: Can Markers Accurately Reflect Large Language Models' Uncertainty?
• Simulating Training Data Leakage in Multiple-Choice Benchmarks for LLM Evaluation
• LittleBit: Ultra Low-Bit Quantization via Latent Factorization
• Diversity of Transformer Layers: One Aspect of Parameter Scaling Laws
• TSENOR: Highly-Efficient Algorithm for Finding Transposable N:M Sparse Masks
• BIRD: Behavior Induction via Representation-structure Distillation
• A Sensitivity-Driven Expert Allocation Method in LoRA-MoE for Efficient Fine-Tuning
• ReplaceMe: Network Simplification via Depth Pruning and Transformer Block Linearization
• Matryoshka Model Learning for Improved Elastic Student Models
• Differential Information Distribution: A Bayesian Perspective on Direct Preference Optimization
• Weight Spectra Induced Efficient Model Adaptation
• MAP: Revisiting Weight Decomposition for Low-Rank Adaptation
• Zero-Shot Vision Encoder Grafting via LLM Surrogates
• MoRE: A Mixture of Low-Rank Experts for Adaptive Multi-Task Learning
• ACE: Exploring Activation Cosine Similarity and Variance for Accurate and Calibration-Efficient LLM Pruning
• SineLoRAΔ: Sine-Activated Delta Compression
• Highly Efficient and Effective LLMs with Multi-Boolean Architectures
• SlimLLM: Accurate Structured Pruning for Large Language Models
• DeCAF: Decentralized Consensus-And-Factorization for Low-Rank Adaptation of Foundation Models
• FireQ: Fast INT4-FP8 Kernel and RoPE-aware Quantization for LLM Inference Acceleration
• DLP: Dynamic Layerwise Pruning in Large Language Models
• LayerIF: Estimating Layer Quality for Large Language Models using Influence Functions
• DenseLoRA: Dense Low-Rank Adaptation of Large Language Models
• QwT-v2: Practical, Effective and Efficient Post-Training Quantization
• Efficient Large Language Model Inference with Neural Block Linearization
• Towards Objective Fine-tuning: How LLMs' Prior Knowledge Causes Potential Poor Calibration?
• MESS+: Dynamically Learned Inference-Time LLM Routing in Model Zoos with Service Level Guarantees
• MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs
• Learning to Select In-Context Demonstration Preferred by Large Language Model
• PiCa: Parameter-Efficient Fine-Tuning with Column Space Projection
• BASE-Q: Bias and Asymmetric Scaling Enhanced Rotational Quantization for Large Language Models
• WINA: Weight Informed Neuron Activation for Accelerating Large Language Model Inference
• GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
• An Empirical Study of Qwen3 Quantization
• RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models
• BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook
• The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation
• Benchmarking Poisoning Attacks against Retrieval-Augmented Generation
• KerZOO: Kernel Function Informed Zeroth-Order Optimization for Accurate and Accelerated LLM Fine-Tuning
• HD-PiSSA: High-Rank Distributed Orthogonal Adaptation
• AuroRA: Breaking Low-Rank Bottleneck of LoRA with Nonlinear Mapping
• How Does Sequence Modeling Architecture Influence Base Capabilities of Pre-trained Language Models? Exploring Key Architecture Design Principles to Avoid Base Capabilities Degradation
• CapBencher: Give Your LLM Benchmark a Built-in Alarm for Test-Set Overfitting
• NeuroTrails: Training with Dynamic Sparse Heads as the Key to Effective Ensembling
• C-LoRA: Contextual Low-Rank Adaptation for Uncertainty Estimation in Large Language Models
• GIM: Improved Interpretability for Large Language Models
• CoMoE: Contrastive Representation for Mixture-of-Experts in Parameter-Efficient Fine-tuning
• TRIM: Achieving Extreme Sparsity with Targeted Row-wise Iterative Metric-driven Pruning
• ScholarBench: A Bilingual Benchmark for Abstraction, Comprehension, and Reasoning Evaluation in Academic Contexts
• Amplify Adjacent Token Differences: Enhancing Long Chain-of-Thought Reasoning with Shift-FFN
• HOFT: Householder Orthogonal Fine-tuning
• Tools in the Loop: Quantifying Uncertainty of LLM Question Answering Systems That Use Tools
• Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuning
• Hierarchical Safety Realignment: Lightweight Restoration of Safety in Pruned Large Vision-Language Models
• RAP: Runtime Adaptive Pruning for LLM Inference
• Revealing Language Model Trajectories via Kullback-Leibler Divergence
• LFTF: Locating First and Then Fine-Tuning for Mitigating Gender Bias in Large Language Models
• On the Generalization vs Fidelity Paradox in Knowledge Distillation
• Social Bias in Popular Question-Answering Benchmarks
• Void in Language Models
• YESciEval: Robust LLM-as-a-Judge for Scientific Question Answering
• Dual Precision Quantization for Efficient and Accurate Deep Neural Networks Inference
• OSoRA: Output-Dimension and Singular-Value Initialized Low-Rank Adaptation
• Breaking Bad Tokens: Detoxification of LLMs Using Sparse Autoencoders
• Dual Decomposition of Weights and Singular Value Low Rank Adaptation
• ABBA-Adapters: Efficient and Expressive Fine-Tuning of Foundation Models
• Safety Alignment Can Be Not Superficial With Explicit Safety Signals
• Fine-tuning Quantized Neural Networks with Zeroth-order Optimization
• Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training and Inference
• GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection
• A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone
• LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades
• Efficient Orthogonal Fine-Tuning with Principal Subspace Adaptation
• F-WANDA: Fisher-Reweighted Post-Training Pruning for Sustainable Deployment of Large Language Models
• ZeroTuning: Unlocking the Initial Token's Power to Enhance Large Language Models Without Training
• A Systematic Analysis of Base Model Choice for Reward Modeling
• Dist2ill: Distributional Distillation for One-Pass Uncertainty Estimation in Large Language Models
• GenKnowSub: Improving Modularity and Reusability of LLMs through General Knowledge Subtraction
• Analog Foundation Models
• Grounding Synthetic Data Evaluations of Language Models in Unsupervised Document Corpora
• GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance
• GPrune-LLM: Generalization-Aware Structured Pruning for Large Language Models
• FloE: On-the-Fly MoE Inference on Memory-constrained GPU
• Towards Explainable Fact Checking
• HiBayES: A Hierarchical Bayesian Modeling Framework for AI Evaluation Statistics
• Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients
• A Switch-Centric In-Network Architecture for Accelerating LLM Inference in Shared-Memory Network
• When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation
• Emotions in the Loop: A Survey of Affective Computing for Emotional Support
• MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance
• Grouped Sequency-arranged Rotation: Optimizing Rotation Transformation for Quantization for Free
• Don't be lazy: CompleteP enables compute-efficient deep transformers
• CREAK: A Dataset for Commonsense Reasoning over Entity Knowledge
• Sphere Retraction Normalizations
• Query Disambiguation via Answer-Free Context: Doubling Performance on Humanity's Last Exam
• ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference
• Brevity Constraints Reverse Performance Hierarchies in Language Models
• AI-Model Network: Concept, Current State and Future
• Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention
• TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts
• Combatting Dimensional Collapse in LLM Pre-Training Data via Diversified File Selection
• HiFloat4 Format for Language Model Inference
• Learning to Plan Before Answering: Self-Teaching LLMs to Learn Abstract Plans for Problem Solving
• Multiagent Protocols with Aggregated Confidence Signals
• Efficient Pre-Training with Token Superposition
• VLA Foundry: A Unified Framework for Training Vision-Language-Action Models
• Large Language Models Generate Harmful Responses Using a Distinct Mechanism, Shared Across Harm Types
• STEM: Scaling Transformers with Embedding Modules
• Nested Learning: The Illusion of Deep Learning Architectures
• Perturbation-efficient Zeroth-order Optimization for Hardware-friendly On-device Training
• R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference
• River-LLM: Large Language Model Seamless Exit Based on KV Share
• Bi-directional Model Cascading with Proxy Confidence
• Parameter Efficiency Is Not Memory Efficiency: Rethinking Fine-Tuning for On-Device LLM Adaptation
• M2XFP: A Metadata-Augmented Microscaling Data Format for Efficient Low-bit Quantization
• Representation-Aware Unlearning via Activation Signatures: From Suppression to Entity-Signature Erasure
• LoCA: Forward-Only LLM Tuning after One-Shot Calibration with Local Credit Assignment
• FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation
• Attention Needs to Focus: A Unified Perspective on Attention Allocation
• Heterogeneity-Aware Microscaling for Efficient Low-Bit LLM Inference
• Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss
• MoEGen: Mixture-of-Experts for Instance-Adaptive LoRA Generation
• Scaling Laws For Scalable Oversight
• Maglev: Sliding Recurrent Memory
• Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes
• Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing Efficiency Through Vocabulary Adaptation
• Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compression
• AROMA: Autonomous Rank-one Matrix Adaptation
• Saliency-driven Dynamic Token Pruning for Large Language Models
• Compass-V2 Technical Report
• Accelerating LLM Inference with Flexible N:M Sparsity via A Fully Digital Compute-in-Memory Accelerator
• Natural Language QA Approaches using Reasoning with External Knowledge
• DIDS: Domain Impact-aware Data Sampling for Large Language Model Training
• Towards Understanding and Improving Refusal in Compressed Models via Mechanistic Interpretability
• It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization
• Activated LoRA: Fine-tuned LLMs for Intrinsics
• LLM-as-a-Judge: Reassessing the Performance of LLMs in Extractive QA
• Unveiling Hidden Collaboration within Mixture-of-Experts in Large Language Models
• Can Pre-training Indicators Reliably Predict Fine-tuning Outcomes of LLMs?
• Enhancing Ultra-Low-Bit Quantization of Large Language Models Through Saliency-Aware Partial Retraining
• Resampling Benchmark for Efficient Comprehensive Evaluation of Large Vision-Language Models
• Can the capability of Large Language Models be described by human ability? A Meta Study
• LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation
• A Neuro-inspired Interpretation of Unlearning in Large Language Models through Sample-level Unlearning Difficulty
• Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptation
• Mosaic: Composite Projection Pruning for Resource-efficient LLMs

Related

• Do Answers to Boolean Questions Need Explanations? Yes
• Frustratingly Easy Natural Question Answering
• ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers
• Reading Comprehension as Natural Language Inference: A Semantic Analysis
• Reasoning-Driven Question-Answering for Natural Language Understanding