Hermes 4 Technical Report

2025/08/25 by Ryan Teknium, Roger Jin, Teknium, Ryan +15 · 3 voices · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #cs.AI

paper · pdf · doi:10.48550/arxiv.2508.18255

arxiv published 2025/08/25 · arxiv updated 2025/09/02

Abstract

We present Hermes 4, a family of hybrid reasoning models that combine structured, multi-turn reasoning with broad instruction-following ability. We describe the challenges encountered during data curation, synthesis, training, and evaluation, and outline the solutions employed to address these challenges at scale. We comprehensively evaluate across mathematical reasoning, coding, knowledge, comprehension, and alignment benchmarks, and we report both quantitative performance and qualitative behavioral analysis. To support open research, all model weights are published publicly at https://huggingface.co/collections/NousResearch/hermes-4-collection-68a731bfd452e20816725728

Citations

• NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model
• Generalizing Verifiable Instruction Following
• Magistral
• rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset
• General-Reasoner: Advancing LLM Reasoning Across All Domains
• Qwen3 Technical Report
• DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning
• FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
• How to Synthesize Text Data without Model Collapse?
• A Comparative Study of Clinical ModernBERT and BioMedical ModernBERT on the DDXPlus Dataset
• Flex Attention: A Programming Model for Generating Optimized Attention Kernels
• Measuring short-form factuality in large language models
• Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World
• TorchTitan: One-stop PyTorch native solution for production ready LLM pre-training
• Generative Reward Models
• AgentInstruct: Toward Generative Teaching with Agentic Flows
• Scaling Synthetic Data Creation with 1,000,000,000 Personas
• The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
• From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline
• DataComp-LM: In search of the next generation of training sets for language models
• MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
• LLM Evaluators Recognize and Favor Their Own Generations
• LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
• SGLang: Efficient Execution of Structured Language Model Programs
• EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models
• GPQA: A Graduate-Level Google-Proof Q&A Benchmark
• Instruction-Following Evaluation for Large Language Models
• Efficient Memory Management for Large Language Model Serving with PagedAttention
• Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
• STaR: Bootstrapping Reasoning With Reasoning
• Measuring Mathematical Problem Solving With the MATH Dataset
• Measuring Massive Multitask Language Understanding
• DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
• Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
• Worst-Case Performance Bounds for Simple One-Dimensional Packing Algorithms

Cited by

• AudioToolAgent: An Agentic Framework for Audio-Language Models

Discussions

• Hermes 4 Technical Report [lobsters, 3 points, 0 comments]
• Hermes 4 Technical Report https://lobste.rs/s/wygxce #ai [bsky, 0 points, 0 comments]
• arxiv.org/abs/2508.18255 [bsky, 0 points, 0 comments]

Related

• Phi-4-reasoning Technical Report
• MTR-Bench: A Comprehensive Benchmark for Multi-Turn Reasoning Evaluation
• From System 1 to System 2: A Survey of Reasoning Large Language Models
• Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance
• Phi-4 Technical Report