Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-up

2024/10/16 by Jiahao Yuan, Yuan, Jiahao, Danxu Du +7 · 4 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2410.12323

openalex publication_date 2024/10/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) have shown remarkable performance in reasoning tasks but face limitations in mathematical and complex logical reasoning. Existing methods to improve LLMs' logical capabilities either involve traceable or verifiable logical sequences that generate more reliable responses by constructing logical structures yet increase computational costs, or introduces rigid logic template rules, reducing flexibility. In this paper, we propose Reversal of Thought (RoT), a plug-and-play and cost-effective reasoning framework designed to enhance the logical reasoning abilities of LLMs during the warm-up phase prior to batch inference. RoT utilizes a Preference-Guided Reverse Reasoning warm-up strategy, which integrates logical symbols for pseudocode planning through meta-cognitive mechanisms and pairwise preference self-evaluation to generate task-specific prompts solely through demonstrations, aligning with LLMs' cognitive preferences shaped by RLHF. Through reverse reasoning, we utilize a Cognitive Preference Manager to assess knowledge boundaries and further expand LLMs' reasoning capabilities by aggregating solution logic for known tasks and stylistic templates for unknown tasks. Experiments across various tasks demonstrate that RoT surpasses existing baselines in both reasoning accuracy and efficiency.

Cited by

• ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLM
• Enhancing Large Language Models for Detecting Mental Manipulation via Annotation-Free Data Augmentation and Anti-Curriculum Distillation
• CoT-Kinetics: A Theoretical Modeling Assessing LRM Reasoning Process
• Fusing Bidirectional Chains of Thought and Reward Mechanisms A Method for Enhancing Question-Answering Capabilities of Large Language Models for Chinese Intangible Cultural Heritage

Related

• Reverse Thinking Makes LLMs Stronger Reasoners
• Reason from Future: Reverse Thought Chain Enhances LLM Reasoning
• Toward Adaptive Reasoning in Large Language Models with Thought Rollback
• Logic-of-Thought: Injecting Logic into Contexts for Full Reasoning in Large Language Models
• Effectively Controlling Reasoning Models through Thinking Intervention