Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions

2024/11/21 by Yu Zhao, Huifeng Yin, Zhao, Yu +15 · 7 voices · 25 citations
Business, Management and Accounting · Computer Science · #Business Process Modeling and Analysis #Multi-Agent Systems and Negotiation #Semantic Web and Ontologies #cs.CL

paper · pdf · doi:10.48550/arxiv.2411.14405

openalex publication_date 2024/11/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Currently OpenAI o1 sparks a surge of interest in the study of large reasoning models (LRM). Building on this momentum, Marco-o1 not only focuses on disciplines with standard answers, such as mathematics, physics, and coding -- which are well-suited for reinforcement learning (RL) -- but also places greater emphasis on open-ended resolutions. We aim to address the question: ''Can the o1 model effectively generalize to broader domains where clear standards are absent and rewards are challenging to quantify?'' Marco-o1 is powered by Chain-of-Thought (CoT) fine-tuning, Monte Carlo Tree Search (MCTS), reflection mechanisms, and innovative reasoning strategies -- optimized for complex real-world problem-solving tasks.

Cited by

• Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR
• LISA: Linear-Indexed Sparse Attention for Efficient Long-Context Reasoning
• Wider or Deeper? Scaling LLM Inference-Time Compute with Adaptive Branching Tree Search
• Evolutionary System 2 Reasoning: An Empirical Proof
• Multilingual Medical Reasoning for Question Answering with Large Language Models
• Reason-Plan-ReAct: A Reasoner-Planner Supervising a ReAct Executor for Complex Enterprise Tasks
• SafeRBench: A Comprehensive Benchmark for Safety Assessment in Large Reasoning Models
• DRIVE: Data Curation Best Practices for Reinforcement Learning with Verifiable Reward in Competitive Code Generation
• MARS-SQL: A multi-agent reinforcement learning framework for Text-to-SQL
• Empowering RepoQA-Agent based on Reinforcement Learning Driven by Monte-carlo Tree Search
• Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generation
• Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism
• Unilaw-R1: A Large Language Model for Legal Reasoning with Reinforcement Learning and Iterative Inference
• Mechanisms of Matter: Language Inferential Benchmark on Physicochemical Hypothesis in Materials Synthesis
• Evaluating Program Semantics Reasoning with Type Inference in System F
• PRIME: Planning and Retrieval-Integrated Memory for Enhanced Reasoning
• CogAtom: From Cognitive Atoms to Olympiad-level Mathematical Reasoning in Large Language Models
• THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoning
• Reasoned Safety Alignment: Ensuring Jailbreak Defense via Answer-Then-Check
• Do Code Semantics Help? A Comprehensive Study on Execution Trace-Based Information for Code Large Language Models
• TORSO: Template-Oriented Reasoning Towards General Tasks
• What Were You Thinking? An LLM-Driven Large-Scale Study of Refactoring Motivations in Open-Source Projects
• mSCoRe: a Multilingual and Scalable Benchmark for Skill-based Commonsense Reasoning
• KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration
• Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models

Discussions

• Alibaba has their own version on GPT-o1. This might be the best description of “o1-type”systems so far arxiv.org/abs/2411.14405 [bsky, 267 points, 8 comments]
• Research into advanced AI reasoning mostly just proves why we all have anxiety arxiv.org/pdf/2411.14405 [bsky, 14 points, 1 comments]
• First there was the COCO dataset (Coco is what my nieces and nephews call me). Now there is this: arxiv.org/abs/2411.14405 Seems like I have a good name for ML. [bsky, 9 points, 0 comments]
• Marco-O1: Towards Open Reasoning Models for Open-Ended Solutions [hn, 2 points, 0 comments]
• "Marco-o1 is powered by Chain-of-Thought (CoT) fine-tuning, Monte Carlo Tree Search (MCTS), reflection mechanisms, and innovative reasoning strategies -- optimized for complex real-world problem-solvi [bsky, 1 points, 0 comments]
• Interesting paper combining SFT and CoT. The reflection prompt seems 'hacky', but ultimately amounts to "Let me double-check that answer" #MLSky arxiv.org/pdf/2411.14405 [bsky, 0 points, 0 comments]
• Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions Alibaba’s Marco-o1 uses Chain-of-Thought tuning and Monte Carlo Tree Search to boost performance. Achieving +6% MGSM gains and outperf [bsky, 0 points, 1 comments]

Related

• OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models
• Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective
• Marco-o1 v2: Towards Widening The Distillation Bottleneck for Reasoning Models
• A Comparative Study on Reasoning Patterns of OpenAI's o1 Model
• REL: Working out is all you need