Contrastive Decoding Improves Reasoning in Large Language Models

2023/09/17 by Sean O'Brien, Mike Lewis, O'Brien, Sean +1 · 1 voice · 14 citations
Computer Science · #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling #cs.AI #cs.CL

paper · pdf · doi:10.48550/arxiv.2309.09117

openalex publication_date 2023/09/17 · openalex created_date 2023/09/20 · openalex updated_date 2026/07/28

Abstract

This record contains the camera-ready paper “LLM Inference in Normal Form: A TRoT-Based Unification.”We show that many inference-time methods for large language models (LLMs)—e.g., MBR/eMBR decoding, Conformal prediction gates, Verifier/Judge ensembles, Self-RAG, Chain/Tree/Graph-of-Thought, and Verifier-Sandwich (VS)—admit a single normal form derived from the Theory of Relativity of Theories (TRoT). Our formulation uses enriched category theory over the Lawvere cost quantale (tropical (min,+) semiring): left/right Kan extensions \Lan/\Ran\Lan/\Ran\Lan/\Ran, residuation (elementwise residuals), masking and nuclei (1-Lipschitz projectors). In this normal form, generation/lifts (K3), safety consolidation (K4), and auditing (K6–K10 RAVE) become GraphBLAS-style sparse operators with explicit stability and auditability contracts. Provable guarantees Implementation equivalence (normal form): common decoding “tricks” map to \Lan\Lan\Lan → Obs → \Ran\Ran\Ran with the same outputs under masking/nucleus conditions. Stability: \Lan\Lan\Lan and the residual are 1-Lipschitz in ℓ∞ℓ_∞ℓ∞. Approximation bounds: geometric truncation error q\*k+11−q\*\fracq_\*^k+11-q_\*1−q\*q\*k+1; soft-min (log-sum-exp) gap k log⁡dmax⁡λcost\frack log d_maxλ_costλcostklogdmax (with join ≡ numeric infimum in Cost polarity). Auditing: a deterministic spending schedule ∑tαt≤αglobal∑_tα_t≤α_global∑tαt≤αglobal with test-(super)martingale e-processes gives time-uniform FWER control; LR/mixture-LR constructions yield valid e-values. Implementation blueprint (GPU/GraphBLAS-ready) \Lan\Lan\Lan: sparse SpMV/SpGEMM on (min,+); \Ran\Ran\Ran: elementwise residual → max-reduce; Numerics: log-domain + row-wise max-shift; masks send forbidden entries to +∞+∞+∞. Reproducibility log (minimum fields): seed; model/revision; tokenizer; prompts; forward map JJJ; transport KKK; masks; λcostλ_costλcost; kkk; effective dmax⁡d_maxdmax; \Lan/\Ran\Lan/\Ran\Lan/\Ran kernel + build flags; hardware/BLAS; e-process updates (et,Mt)(e_t,M_t)(et,Mt); Conformal calibration snapshots.

Citations

• Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity

Cited by

• Enhancing Instruction-Following Capabilities in Seq2Seq Models: DoLA Adaptations for T5
• LightReasoner: Can Small Language Models Teach Large Language Models Reasoning?
• Exploring and Mitigating Fawning Hallucinations in Large Language Models
• TRACEALIGN -- Tracing the Drift: Attributing Alignment Failures to Training-Time Belief Sources in LLMs
• Reading the Finetuning Prior: Verbatim Content Recovery via Contrastive Decoding Diffing
• Don't Mix Rewards, Mix Policies: Policy Decomposition and Optimization for Multi-Reward RL
• DecoRTL: A Run-time Decoding Framework for RTL Code Generation with LLMs
• CROP: Circuit Retrieval and Optimization with Parameter Guidance using LLMs
• UCD: Unlearning in LLMs via Contrastive Decoding
• Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMs
• RAST: Reasoning Activation in LLMs via Small-model Transfer
• ProxyThinker: Test-Time Guidance through Small Visual Reasoners
• Spotlight Your Instructions: Instruction-following with Dynamic Attention Steering
• UniDetox: Universal Detoxification of Large Language Models via Dataset Distillation
• Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents

Discussions

• Contrastive decoding improves reasoning in large language models [hn, 21 points, 1 comments]

Related

• Contrastive Decoding: Open-ended Text Generation as Optimization
• Large Language Models are Contrastive Reasoners
• Distillation Contrastive Decoding: Improving LLMs Reasoning with Contrastive Decoding and Distillation
• Explaining and Improving Contrastive Decoding by Extrapolating the Probabilities of a Huge and Hypothetical LM
• Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generation