Mixture of Reasonings: Teach Large Language Models to Reason with Adaptive Strategies

2025/07/01 by Xiong, Tao, Hu, Xavier, Fan, Wenyan +1 · 1 citation
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2507.00606

Abstract

Large language models (LLMs) excel in complex tasks through advanced prompting techniques like Chain-of-Thought (CoT) and Tree-of-Thought (ToT), but their reliance on manually crafted, task-specific prompts limits adaptability and efficiency. We introduce Mixture of Reasoning (MoR), a training framework that embeds diverse reasoning strategies into LLMs for autonomous, task-adaptive reasoning without external prompt engineering. MoR has two phases: Thought Generation, creating reasoning chain templates with models like GPT-4o, and SFT Dataset Construction, pairing templates with benchmark datasets for supervised fine-tuning. Our experiments show that MoR significantly enhances performance, with MoR150 achieving 0.730 (2.2% improvement) using CoT prompting and 0.734 (13.5% improvement) compared to baselines. MoR eliminates the need for task-specific prompts, offering a generalizable solution for robust reasoning across diverse tasks.

Cited by

• From Efficiency to Adaptivity: A Deeper Look at Adaptive Reasoning in Large Language Models

Related

• Enhancing the Reasoning Capabilities of Small Language Models via\n Solution Guidance Fine-Tuning
• Breaking Thought Patterns: A Multi-Dimensional Reasoning Framework for LLMs
• Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoning
• Think Beyond Size: Adaptive Prompting for More Effective Reasoning
• Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought