Efficient Memory Management for Large Language Model Serving with PagedAttention

2023/09/12 by Woosuk Kwon, Zhuohan Li, Kwon, Woosuk +15 · 4 voices · 1352 citations
Computer Science · #Caching and Content Delivery #Software System Performance and Reliability #Topic Modeling #cs.DC #cs.LG

paper · pdf · doi:10.48550/arxiv.2309.06180

openalex publication_date 2023/09/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

High throughput serving of large language models (LLMs) requires batching sufficiently many requests at a time. However, existing systems struggle because the key-value cache (KV cache) memory for each request is huge and grows and shrinks dynamically. When managed inefficiently, this memory can be significantly wasted by fragmentation and redundant duplication, limiting the batch size. To address this problem, we propose PagedAttention, an attention algorithm inspired by the classical virtual memory and paging techniques in operating systems. On top of it, we build vLLM, an LLM serving system that achieves (1) near-zero waste in KV cache memory and (2) flexible sharing of KV cache within and across requests to further reduce memory usage. Our evaluations show that vLLM improves the throughput of popular LLMs by 2-4× with the same level of latency compared to the state-of-the-art systems, such as FasterTransformer and Orca. The improvement is more pronounced with longer sequences, larger models, and more complex decoding algorithms. vLLM's source code is publicly available at https://github.com/vllm-project/vllm

Cited by

• TENT: A Declarative Slice Spraying Engine for Performant and Resilient Data Movement in Disaggregated LLM Serving
• Beyond Text-to-SQL: Can LLMs Really Debug Enterprise ETL SQL?
• LMEB: Long-horizon Memory Embedding Benchmark
• Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning
• Learning as Reasoning Unfolds: Progressive Rollout Allocation for Efficient Reinforcement Learning
• Mapping Political-Elite Networks in Europe with a Multilingual Joint Entity-Relation Extraction Pipeline
• SAFE-Pruner: Semantic Attention-Guided Future-Aware Token Pruning for Efficient Vision-Language-Action Manipulation
• Fewer Paths, Better Performance: Understanding the ZCube Topology through Braess's Paradox
• Agentic CPU-GPU Scheduling for Heterogeneous AI Workloads
• Adversarial Prompts for Acceptance Collapse in Speculative Decoding
• Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judging
• Learning to Reason for Factuality
• HiKV: Hierarchical Importance-Aware KV Cache with Hardware Acceleration for LLM Decoding
• Persistent Computational State: A Session-Centric Runtime for Generative World Models
• A Unified Moral-Value Dataset for Instruction Tuning
• FlashDiff: Efficient Regional Execution and Scheduling for Diffusion Model Serving
• Efficient Clustering with Provable Guardrails for LLM Inference at Scale
• PReM: Learning What to Preserve and When to Refresh for Context Compression
• HijackKV: New Threat in Position-Independent KV Cache Reuse
• GPU-to-Grid: Voltage Regulation via GPU Utilization Control
• An LLM-powered Agentic Recommendation System for Connected TV Content Discovery
• Harmonia: Algorithm-Hardware Co-Design for Memory- and Compute-Efficient BFP-based LLM Inference
• Refusal-Gated Decoding: Preserving Refusal Behavior Under High-Temperature Sampling
• OntoBook: Ontology-Grounded Synthetic Textbooks for Medical Encoder Pretraining
• Contrastive On-Policy Distillation
• Structured Feedback Improves Repair in an LLM Agent Loop
• Vocabulary Dropout for Curriculum Diversity in LLM Co-Evolution
• PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference
• Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning
• The Price of Reasoning: Cost-Quality Tradeoffs in Reinforcement Learning for Neural Machine Translation
• DocOps: A Verifiable Benchmark for Autonomous Agents in Complex Document Operations
• How Fast Can Reward Models Score? A Systems Study of C++ and PyTorch Inference Runtimes for RLHF
• Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing
• BLUE: Semantics-Preserving Video Compression for Efficient Vision-Language Surveillance Analytics
• Timeripple: Accelerating vDiTs by Understanding the Spatio-Temporal Correlations in Latent Space
• AnythingReality: Robust Online Gaussian Splatting SLAM for Open-Vocabulary VR Scene Exploration
• FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications
• Robust Reasoning Benchmark
• Fluid Reasoning Representations
• Efficient Multi-round LLM Inference over Disaggregated Serving
• Harness TTS: Towards Context-Aware Expressive Speech Synthesis with Harness Layer
• Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning
• AgentJet: A Distributed Swarm Training Framework for Agentic Reinforcement Learning
• AI Tour Meeting: Group Travel Planning by LLM Agents
• Keeping the Cache Warm Pays: Keepalive Economics for Agentic Workloads
• DAIS: Dependency-Aware Intermediate QA Supervision for Complex Reasoning
• Large Language Models Explore by Latent Distilling
• InstantInfer: Enabling Fast LLM Cold Start with Communicating Finite Automata
• HACO: Hedged Agent Computing for Reliable LLM Systems
• ARMOR: Stabilizing On-Policy LLM RL with Off-Policy Anchor Samples
• Reasoning Before Translation: Enhancing Legal Machine Translation with Structured Reasoning
• Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information
• D-NOVA: In-Storage Retrieval Accelerator via Dual-Bound 3D NAND-Optimized Similarity Search with Vector Adaptation
• Natural Language Access to Domain-Specific Metadata: A Reusable Framework for LLM Query Generation
• What Transfers Under Source Shift? Definitions, Examples, and Fine-Tuning for Climate Disclosure Classification
• Harness Engineering for LLM-Driven GPU Kernel Generation
• Search-on-Graph-R1: Training Large Language Models to Search Knowledge Graphs with Reinforcement Learning
• Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling
• Don't Blame the Large Language Model: How Agent Harness Evolution Shapes Coding Agent Quality
• HyMCache: A KV Cache Framework for Multi-Turn LLM Serving with CXL-Hybrid Memory
• Capability from Access Structure, Not Scale: Lower Bounds and Pre-Registered Tests for Hybrid Sequence Models
• Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches
• Operational Proto-Introspection in Looped Language Models: Process-Quality Taps, Executable Branching, and the Readout-Control Boundary
• Decode-Time Grammars: Constrained LLM Generation over a Refinement Order of Grammar Fragments
• Hardware Mechanisms to Dynamically Throttle AI Performance
• DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training
• When Does Muon Help Agentic Reinforcement Learning?
• A Systematic Investigation of RL-Jailbreaking in LLMs
• SlotGuard: Stop Oversharing Private Local Context in LLM Agent Transcri
• Lossless but Not Free: An Empirical Anatomy of Speculative Decoding on Consumer Hardware
• ThAME: 3D Memory-Enabled Heterogeneous Accelerator for LLM Mixture of Experts
• Talaria: Session-Aware Serverless Serving of Hundred-Billion-Parameter LLMs
• Transition-Aware Backend Dispatch for Edge LLM Inference
• DAN-Scheduler: Deterministic Three-Stage Co-Optimization of Scheduling, Memory Layout, and Pipeline Overlap for General-Purpose NPUs
• Do Value Vectors in Deep Layers Need Context from the Residual Stream?
• MMGraphRAG: Bridging Vision and Language with Interpretable Multimodal Knowledge Graphs
• Is "Knowing It's Malicious Enough?" Evaluating LLMs for Fine-Grained Malware Behavior Auditing
• Robust KV Cache Management for LLM Serving under Output Token Length Uncertainty
• CHIA: An open-source framework for principled, agentic AI-driven hardware/software co-design research
• Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR
• Constraint-Driven Model Optimization: An Industry Framework for Selecting Compression and Acceleration Techniques in Modern Machine Learning Systems
• Scalable LLM Agent Tool Access in the Cloud
• PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization
• When Do Multi-Agent Systems Help? An Information Bottleneck Perspective
• DualKV: Shared-Prompt Flash Attention for Efficient RL Training with Large Rollouts and Long Contexts
• Enabling Spatially Fine-Grained DVFS in Neural Processing Units for Energy-Efficient LLM Serving
• RIMS: Preference Optimization via Smoothed Multi-pair Aggregation for Small-Scale LLM Retrieval-Augmented Generation
• Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
• Speculative Decoding with a Speculative Vocabulary
• Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives
• Length Penalties Make Chain-of-Thought Less Monitorable
• SODA: Semi On-Policy Black-Box Distillation for Large Language Models
• Auto-Scaling Heterogeneous Neural Processing Units for Energy and Cost-Efficient LLM Serving
• JoyNexus: Service-Oriented Multi-Tenant Post-Training for VLA Models
• KD-Judge: A Knowledge-Driven Automated Judge Framework for Functional Fitness Movements on Edge Devices
• MemTrace: Tracing and Attributing Errors in Large Language Model Memory Systems
• Attention to Detail: Evaluating Energy, Performance, and Accuracy Trade-offs Across vLLM Configurations
• On-Policy Delta Distillation
• Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models
• Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent
• Reflex: Real-Time VLA Control through Streaming Inference
• Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers
• Models Can Model, But Can't Bind: Structured Grounding in Text-to-Optimization
• Reliability-Aware LLM Alignment from Inconsistent Human Feedback
• Self-Compacting Language Model Agents
• One Student, Many Teachers: Multi-Task On-Policy Distillation via Soft-Prompt Privileged Context
• High-accuracy Low-Bit KV-Cache Quantization via Local Distribution Restoration
• CIGPO: Contextual Information-Gain Policy Optimization for Multi-Turn Evidence-Reading LLM Agents
• Still: Amortized KV Cache Compaction in a Single Forward Pass
• Workload-Aware Caching for Multi-Agent Systems
• SonicSampler: Unified Tile-Aware Kernels for LLM Sampling and Speculative Verification
• DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions
• Benchmarking Confidential GPU Inference on NVIDIA H100 under Intel TDX
• InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents
• CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs
• JAXBench: Benchmarking Autonomous TPU Kernel Optimization
• CPPL: A Circuit Prompt Programming Language
• Decoupled Attention Fusion: Accelerating RAG with Efficient KV Cache Reuse
• AgentKVShift: Efficient KV Cache Reuse for Agentic Memory Systems
• PEARL: Solver-in-the-Loop Interactive Optimization Modeling from Natural Language
• Domyn-Small: A European 10B Reasoning Language Model
• Beyond Accuracy and Cost: Latency-Aware LLM Query Routing for Dynamic Workloads
• Transferable Latency Prediction for Fast LLM Screening on Heterogeneous Edge Devices
• Embarrassingly Simple Self-Distillation Improves Code Generation
• Sequential KV Cache Compression via Probabilistic Language Tries: Beyond the Per-Vector Shannon Limit
• Effective Distillation to Hybrid xLSTM Architectures
• Mamba-3: Improved Sequence Modeling using State Space Principles
• Can LLMs Beat Classical Hyperparameter Optimization Algorithms? A Study on autoresearch
• Flash-KMeans: Fast and Memory-Efficient Exact K-Means
• LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models
• Speculative Speculative Decoding
• Learning to Reason in 13 Parameters
• PrAg-PO: Prompt Augmented Policy Optimization for Robust and Diverse Mathematical Reasoning
• Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?
• Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Use
• Strategies for Span Labeling with Large Language Models
• Leveraging External Knowledge for Historical Document Restoration via Retrieval-Augmented Large Language Models
• Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem
• NL2Logic: AST-Guided Translation of Natural Language into First-Order Logic with Large Language Models
• In-Context Probing for Membership Inference in Fine-Tuned Language Models
• PHOTON: Hierarchical Autoregressive Modeling for Lightspeed and Memory-Efficient Language Generation
• LLM Output Drift: Cross-Provider Validation & Mitigation for Financial Workflows
• Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models
• Not All Bits Are Equal: Scale-Dependent Memory Optimization Strategies for Reasoning Models
• Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution
• Base Models Know How to Reason, Thinking Models Learn When
• Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs
• A Cost-Benefit Analysis of On-Premise Large Language Model Deployment: Breaking Even with Commercial LLM Services
• A Survey on Diffusion Language Models
• Reinforcement Learning Teachers of Test Time Scaling
• ProxAnn: Use-Oriented Evaluations of Topic Models and Document Clustering
• Self-Adapting Language Models
• Log-Linear Attention
• RAVENEA: A Benchmark for Multimodal Retrieval-Augmented Visual Culture Understanding
• Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers.
• Using Reinforcement Learning to Train Large Language Models to Explain Human Decisions
• M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models
• SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion
• ZSMerge: Zero-Shot KV Cache Compression for Memory-Efficient Long-Context LLMs
• Multi-Modal Framing Analysis of News
• InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU
• Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs
• Is That Your Final Answer? Test-Time Scaling Improves Selective Question Answering
• CalibQuant: 1-Bit KV Cache Quantization for Multimodal LLMs
• Scaling Laws in Patchification: An Image Is Worth 50,176 Tokens And More
• DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
• s1: Simple test-time scaling
• Accelerating Time Series Foundation Models with Speculative Decoding
• Remedy-R: Generative Reasoning for Machine Translation Evaluation without Error Annotations
• DMA: Online RAG Alignment with Human Feedback
• C8s: A Confidential Kubernetes Architecture
• Discourse Diversity in Multi-Turn Empathic Dialogue
• Style Amnesia: Investigating Speaking Style Degradation and Mitigation in Multi-Turn Spoken Language Models
• LiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillation
• NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models
• Reinforcement Learning via Self-Distillation
• Replay Failures as Successes: Sample-Efficient Reinforcement Learning for Instruction Following
• MindWatcher: Toward Smarter Multimodal Tool-Integrated Reasoning
• Trust Region Masking for Long-Horizon LLM Reinforcement Learning
• Viability and Performance of a Private LLM Server for SMBs: A Benchmark Analysis of Qwen3-30B on Consumer-Grade Hardware
• Fleet: Hierarchical Task-based Abstraction for Megakernels on Multi-Die GPUs
• Argus: Token Aware Distributed LLM Inference Optimization
• Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulation
• OptiNIC: A Resilient and Tail-Optimal RDMA NIC for Distributed ML Workloads
• WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference
• Agentic Permissions Policy Algebra for Taint Confinement in LLM Agents
• Self-Rewarded Multimodal Coherent Reasoning Across Diverse Visual Domains
• RollArt: Disaggregated Multi-Task Agentic RL Training at Scale
• Role-Based Fault Tolerance System for LLM RL Post-Training
• AFA-LoRA: Enabling Non-Linear Adaptations in LoRA with Activation Function Annealing
• Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving
• Efficient Multi-Model Orchestration for Self-Hosted Large Language Models
• VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning
• Masking Teacher and Reinforcing Student for Distilling Vision-Language Models
• MatKV: Trading Compute for Flash Storage in LLM Inference
• DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention
• Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory
• SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving
• Language Shapes Instruction Hierarchy Compliance in Multilingual LLMs
• Do LLM Debates Repeat Arguments Differently Across Languages?
• LiteTopK: Exploiting the Curse of Dimensionality for a Fused Indexer-TopK Kernel in Long-Context Sparse Attention
• StreamIndex: Memory-Bounded Compressed Sparse Attention via Streaming Top-k
• In Search of the Ingredients of Open-Endedness: Replicating Picbreeder with Large Vision-Language Models
• Accelerating Language Model Workflows with Prompt Choreography
• A Frozen 12B Beats Frontier Models on Verified Work: 100% Accuracy, 0 Tokens, Bit-Exact, Forever
• Kalypso: Relational LLM Serving
• INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models
• KAP: Bridging the Knowledge Selection-Runtime Consumption Gap in LLM Systems
• LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decoding
• Decoding the Skew: Distribution-Aware MoE Inference with Adaptive Kernel Dispatch
• Memory for Large Language Models
• Everyone is unique: Towards Behaviorally Heterogeneous Negotiation Dialogue Systems for Debt Collection
• CoSA: Accelerating Long-Context Inference via Proxy-Kernel Co-Designed Sparse Attention
• Beyond Prefill-Decode Disaggregation: Dissecting LLM Inference for Heterogeneous Platforms via Dynamic Operator Scheduling
• Conformal Cascade: Distribution-Free Accuracy Guarantees for Multi-Tier LLM Inference
• LLM as Forecasting Planner: Training-Free Text Conditioning for Time-Series Foundation Models
• Gleam: Adaptive Network-Efficient CUDA API Remoting for Cross-Device GPU Sharing over LANs
• MixQuant: Adaptive Mixed-Precision Quantization for Large Language Models
• MedJudgeRAG: Option-Wise Evidence Judgment with Dynamic Knowledge Graphs for Medical MCQA
• The Best Programming Language for Tokenmaxxing: An Investigation of Coding Agent Behavior Across Programming Languages
• AssumptionMiner: Extracting, Tracing, and Revising Implicit Assumptions in LLM Code Generation
• Agent Team Work Zone: An Automated, Persistent Workspace for Long-Lived Claude Code Agent Teams
• TGMS: An Agent-Native Bi-Temporal Graph Management System
• DRC-Aid: Design-Rule Correction via Agentic Framework utilizing Inference-Time Large Language Models
• AdaKP: Online Adaptive Knowledge-Point Selection for Reasoning-Oriented Reinforcement Learning
• MPR-CiteG: Enhancing RAG with Multi-Portfolio Retrieval and Citation-Grounded Generation
• StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design
• How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift
• FARM: Find Anything using Relational Spatial Memory
• PTStore (Prefix Tensor Store): Distributed Prefix Caching and Replication for High Throughput Inference Serving
• TRACE: Business Rule-Grounded Reasoning Curriculum for Knowledge-Preserving Parametric Tool Retrieval in Enterprise LLMs
• HeraSys: Collaborative Serving of Multiple LLM Workflows via Fine-Grained End-to-End Optimization
• Evolving from Lessons: Skill-Augmented Table Graph Reasoning for Operation-wise Table Question Answering
• DynaResize: Runtime GPU Reallocation for Disaggregated LLM Post-Training
• HyCE-RAG: Hypergraph Chain-of-Evidence Retrieval-Augmented Generation for Explainable Multi-hop Question Answering
• xMIx: High-Performance Serving-Time Platform for Mechanistic Interpretability Apps
• Lexical discovery in unknown environments orchestrated by Large Language Models
• Keyword Matters: Unveiling the Energy Sensitivity of On-Device LLM Prompting
• Codifying the Judge: Scalable Evaluation via Program Distillation
• Procedural Knowledge at Scale Improves Reasoning
• Geometric Context Transformer for Streaming 3D Reconstruction
• SkipOPU: An FPGA-based Overlay Processor for Large Language Models with Dynamically Allocated Computation
• Multi-Task GRPO: Reliable LLM Reasoning Across Tasks
• Adapter Merging Reactivates Latent Reasoning Traces: A Mechanism Analysis
• Entropy Sentinel: Probing Entropy Traces for LLM Monitoring
• LIME:Accelerating Collaborative Lossless LLM Inference on Memory-Constrained Edge Devices
• A Comedy of Estimators: On KL Regularization in RL Training of LLMs
• Rethinking Sample Polarity in Reinforcement Learning with Verifiable Rewards
• Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism
• RLLaVA: An RL-central Framework for Language and Vision Assistants
• A Plan Reuse Mechanism for LLM-Driven Agent
• One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agents
• ClarifyMT-Bench: Benchmarking and Improving Multi-Turn Clarification for Conversational Large Language Models
• EVE: A Generator-Verifier System for Generative Policies
• Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
• RHAPSODY: Execution of Hybrid AI-HPC Workflows at Scale
• AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agent
• Cube Bench: A Benchmark for Spatial Visual Reasoning in MLLMs
• Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs
• Toward Explaining Large Language Models in Software Engineering Tasks
• Learning to Reason in LLMs by Expectation Maximization
• Predictive-LoRA: A Proactive and Fragmentation-Aware Serverless Inference System for LLMs
• CodeSimpleQA: Scaling Factuality in Code Large Language Models
• RAPID-LLM: Resilience-Aware Performance analysis of Infrastructure for Distributed LLM Training and Inference
• MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoning
• CascadeInfer: Length-Aware Scheduling of LLM Serving with Low Latency and Load Balancing
• On Finding Inconsistencies in Documents
• X-Talk: On the Underestimated Potential of Modular Speech-to-Speech Dialogue System
• SmartSight: Mitigating Hallucination in Video-LLMs Without Compromising Video Understanding via Temporal Attention Collapse
• TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale
• SRS-Stories: Vocabulary-constrained multilingual story generation for language learning
• Efficient Mixture-of-Agents Serving via Tree-Structured Routing, Adaptive Pruning, and Dependency-Aware Prefill-Decode Overlap
• Layout-Aware Text Editing for Efficient Transformation of Academic PDFs to Markdown
• Shuttling Compiler for Trapped-Ion Quantum Computers Based on Large Language Models
• When the Gold Standard Isn't Necessarily Standard: Challenges of Evaluating the Translation of User-Generated Content
• Enabling Disaggregated Multi-Stage MLLM Inference via GPU-Internal Scheduling and Resource Sharing
• KV Admission: Learning What to Write for Efficient Long-Context Inference
• CodeDance: A Dynamic Tool-integrated MLLM for Executable Visual Reasoning
• Understanding Generalization in Role-Playing Models via Information Theory
• Enhancing Long Document Long Form Summarisation with Self-Planning
• Trust-Region Adaptive Policy Optimization
• Reinforcement Learning for Self-Improving Agent with Skill Library
• A Solver-in-the-Loop Framework for Improving LLMs on Answer Set Programming for Logic Puzzle Solving
• XLM: A Python package for non-autoregressive language models
• AdaTooler-V: Adaptive Tool-Use for Images and Videos
• Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning
• A Benchmark and Agentic Framework for Omni-Modal Reasoning and Tool Use in Long Videos
• AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning
• MEPIC: Memory Efficient Position Independent Caching for LLM Serving
• Exploration of Augmentation Strategies in Multi-modal Retrieval-Augmented Generation for the Biomedical Domain: A Case Study Evaluating Question Answering in Glycobiology
• DataFlow: An LLM-Driven Framework for Unified Data Preparation and Workflow Automation in the Era of Data-Centric AI
• AI4EOSC: a Federated Cloud Platform for Artificial Intelligence in Scientific Research
• LoPA: Scaling dLLM Inference via Lookahead Parallel Decoding
• Scaling Text2SQL via LLM-efficient Schema Filtering with Functional Dependency Graph Rerankers
• MultiPath Transfer Engine: Breaking GPU and Host-Memory Bandwidth Bottlenecks in LLM Services
• INTELLECT-3: Technical Report
• The Perceptual Observatory Characterizing Robustness and Grounding in MLLMs
• Dynamic Rebatching for Efficient Early-Exit Inference with DREX
• Optimizing Agentic Language Model Inference via Speculative Tool Calls
• Towards Proactive Personalization through Profile Customization for Individual Users in Dialogues
• How Much is Too Much? Exploring LoRA Rank Trade-offs for Retaining Knowledge and Domain Robustness
• Bolmo: Byteifying the Next Generation of Language Models
• CTkvr: KV Cache Retrieval for Long-Context LLMs via Centroid then Token Indexing
• DEER: Draft with Diffusion, Verify with Autoregressive Models
• Mixture of Attention Schemes (MoAS): Learning to Route Between MHA, GQA, and MQA
• Spatia: Video Generation with Updatable Spatial Memory
• EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving
• ViRC: Enhancing Visual Interleaved Mathematical CoT with Reason Chunking
• Ladder Up, Memory Down: Low-Cost Fine-Tuning With Side Nets
• Cornfigurator: Automated Planning for Any-to-Any Multimodal Model Serving
• Sharing State Between Prompts and Programs
• SAGE: Training Smart Any-Horizon Agents for Long Video Reasoning with Reinforcement Learning
• VLCache: Computing 2% Vision Tokens and Reusing 98% for Vision-Language Inference
• Async Control: Stress-testing Asynchronous Control Measures for LLM Agents
• From User Interface to Agent Interface: Efficiency Optimization of UI Representations for LLM Agents
• Beyond Training: Enabling Self-Evolution of Agents with MOBIMEM
• Fine-Grained Energy Prediction For Parallellized LLM Inference With PIE-P
• Revisiting the Reliability of Language Models in Instruction-Following
• ProServe: Unified Multi-Priority Request Scheduling for LLM Serving
• Persistent Personas? Role-Playing, Instruction Following, and Safety in Extended Interactions
• gpu_ext: Extensible OS Policies for GPUs via eBPF
• Exact Flow Linear Attention: Exact Solution from Continuous-Time Dynamics
• Coupled Variational Reinforcement Learning for Language Model General Reasoning
• More Than the Final Answer: Improving Visual Extraction and Logical Consistency in Vision-Language Models
• HetRL: Efficient Reinforcement Learning for LLMs in Heterogeneous Environments
• Taint-Based Code Slicing for LLMs-based Malicious NPM Package Detection
• Training Versatile Coding Agents in Synthetic Environments
• The Instability of Safety: How Random Seeds and Temperature Expose Inconsistent LLM Refusal Behavior
• Instruction-Tuning Open-Weight Language Models for BPMN Model Generation
• xGR: Efficient Generative Recommendation Serving at Scale
• AgentBalance: Backbone-then-Topology Design for Cost-Effective Multi-Agent Systems under Budget Constraints
• AGAPI-Agents: An Open-Access Agentic AI Platform for Accelerated Materials Design on AtomGPT.org
• Speculative Decoding Speed-of-Light: Optimal Lower Bounds via Branching Random Walks
• Asynchronous Reasoning: Training-Free Interactive Thinking LLMs
• CXL-SpecKV: A Disaggregated FPGA Speculative KV-Cache for Datacenter LLM Serving
• ESS: An Offload-Centric Latent-Cache Management Architecture for DeepSeek-V3.2-Exp
• LLM-Auction: Generative Auction towards LLM-Native Advertising
• Unlocking the Address Book: Dissecting the Sparse Semantic Structure of LLM Key-Value Caches via Sparse Autoencoders
• InFerActive: Interactive Tree-Based Exploration of LLM Sampling for Safety Evaluation
• SWAA: Sliding Window Attention Adaptation for Efficient and Quality Preserving Long Context Processing
• KBQA-R1: Reinforcing Large Language Models for Knowledge Base Question Answering
• Supporting Dynamic Agentic Workloads: How Data and Agents Interact
• RouteRAG: Efficient Retrieval-Augmented Generation from Text and Graph via Reinforcement Learning
• WarmServe: Enabling One-for-Many GPU Prewarming for Multi-LLM Serving
• ODMA: On-Demand Memory Allocation Framework for LLM Serving on LPDDR-Class Accelerators
• RACAM: Enhancing DRAM with Reuse-Aware Computation and Automated Mapping for ML Inference
• GoodSpeed: Optimizing Fair Goodput with Adaptive Speculative Decoding in Distributed Edge Inference
• Same Content, Different Answers: Cross-Modal Inconsistency in MLLMs
• LaMoSys3.5D: Enabling 3.5D-IC-Based Large Language Model Inference Serving Systems via Hardware/Software Co-Design
• To Think or Not to Think: The Hidden Cost of Meta-Training with Excessive CoT Examples
• QSTN: A Modular Framework for Robust Questionnaire Inference with Large Language Models
• Magneton: Optimizing Energy Efficiency of ML Systems via Differential Energy Debugging
• SoMe: A Realistic Benchmark for LLM-based Social Media Agents
• Mind to Hand: Purposeful Robotic Control via Embodied Reasoning
• SkipKV: Selective Skipping of KV Generation and Storage for Efficient Inference with Large Reasoning Models
• Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning
• Decouple to Generalize: Context-First Self-Evolving Learning for Data-Scarce Vision-Language Reasoning
• MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learning
• PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory
• RLAX: Large-Scale, Distributed Reinforcement Learning for Large Language Models on TPUs
• VG-Refiner: Towards Tool-Refined Referring Grounded Reasoning via Agentic Reinforcement Learning
• Policy-based Sentence Simplification: Replacing Parallel Corpora with LLM-as-a-Judge
• A-3PO: Accelerating Asynchronous LLM Training with Staleness-aware Proximal Policy Approximation
• Compass: Co-Exploration of Mapping and Hardware for Heterogeneous Multi-Chiplet Accelerators Targeting LLM Inference Service Workloads
• Bootstrapping Fuzzers for Compilers of Low-Resource Language Dialects Using Language Models
• RoBoN: Routed Online Best-of-n for Test-Time Scaling with Multiple LLMs
• PrivCode: When Code Generation Meets Differential Privacy
• Structured Document Translation via Format Reinforcement Learning
• RefineBench: Evaluating Refinement Capability of Language Models via Checklists
• Distilling Temporal Search and Reasoning: Evolving LLMs for Future Prediction via Harness-Assisted Efficient Data Synthesis
• LAWS: Learning from Actual Workloads Symbolically -- A Self-Certifying Parametrized Cache Architecture for Neural Inference, Robotics, and Edge Deployment
• Beyond Factual Accuracy: Evaluating Global Reasoning Integrity in RAG Systems with LogicScore
• Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
• CryptoTensors: A Light-Weight Large Language Model File Format for Highly-Secure Model Distribution
• Solving LLM Repetition Problem in Production: A Comprehensive Study of Multiple Solutions
• Distance Is All You Need: Radial Dispersion for Uncertainty Estimation in Large Language Models
• KV Cache Recycling to Expand Usable Context Capacity in Low Parameter LLMs
• Log Probability Tracking of LLM APIs
• AR-Med: Automated Relevance Enhancement in Medical Search via LLM-Driven Information Augmentation
• KVNAND: Efficient On-Device Large Language Model Inference Using DRAM-Free In-Flash Computing
• OpenTrack3D: Towards Accurate and Generalizable Open-Vocabulary 3D Instance Segmentation
• AsymPuzl: An Asymmetric Puzzle for multi-agent cooperation
• TokenScale: Timely and Accurate Autoscaling for Disaggregated LLM Serving with Token Velocity
• Continuous Prompts: LLM-Augmented Pipeline Processing over Unstructured Streams
• Leveraging LLMs for Structured Data Extraction from Unstructured Patient Records
• Large Language Models as Generalist Policies for Network Optimization
• Divide, then Ground: Adapting Frame Selection to Query Types for Long-Form Video Understanding
• SPARK: Stepwise Process-Aware Rewards for Reference-Free Reinforcement Learning
• TokenPowerBench: Benchmarking the Power Consumption of LLM Inference
• Fairy2i: Training Complex LLMs from Real LLMs with All Parameters in \± 1, ± i\
• MindGPT-4ov: An Enhanced MLLM via a Multi-Stage Post-Training Paradigm
• PaCo-RL: Advancing Reinforcement Learning for Consistent Image Generation with Pairwise Reward Modeling
• GeoViS: Geospatially Rewarded Visual Search for Remote Sensing Visual Grounding
• Cross-Lingual Prompt Steerability: Towards Accurate and Robust LLM Behavior across Languages
• VACoT: Rethinking Visual Data Augmentation with VLMs
• SpecPV: Improving Self-Speculative Decoding for Long-Context Generation via Partial Verification
• promptolution: A Unified, Modular Framework for Prompt Optimization
• Think in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoning
• Trinity: Disaggregating Vector Search from Prefill-Decode Disaggregation in LLM Serving
• PAI-Bench: A Comprehensive Benchmark For Physical AI
• KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference
• Rectifying LLM Thought from Lens of Optimization
• A Systematic Characterization of LLM Inference on GPUs
• MAC-SLU: Multi-Intent Automotive Cabin Spoken Language Understanding Benchmark
• MCAT: Scaling Many-to-Many Speech-to-Text Translation with MLLMs to 70 Languages
• Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding
• ZO-ASR: Zeroth-Order Fine-Tuning of Speech Foundation Models without Back-Propagation
• KVReviver: Reversible KV Cache Compression with Sketch-Based Token Reconstruction
• Chain of Unit-Physics: A Primitive-Centric Approach to Scientific Code Synthesis
• Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning
• Bias Injection Attacks on RAG Databases and Sanitization Defenses
• SpeContext: Enabling Efficient Long-context Reasoning with Speculative Context Sparsity in LLMs
• SIMPLE: Disaggregating Sampling from GPU Inference into a Decision Plane for Faster Distributed LLM Serving
• ESPO: Entropy Importance Sampling Policy Optimization
• Beyond Code Pairs: Dialogue-Based Data Generation for LLM Code Translation
• Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models
• ThetaEvolve: Test-time Learning on Open Problems
• OctoMed: Data Recipes for State-of-the-Art Multimodal Medical Reasoning
• MathSight: A Benchmark Exploring Have Vision-Language Models Really Seen in University-Level Mathematical Reasoning?
• ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answering
• ORION: Teaching Language Models to Reason Efficiently in the Language of Thought
• Behavior-Equivalent Token: Single-Token Replacement for Long Prompts in LLMs
• CacheTrap: Injecting Trojans in LLMs without Leaving any Traces in Inputs or Weights
• Geometrically-Constrained Agent for Spatial Reasoning
• DisCEdge: Distributed Context Management for Large Language Models at the Edge
• OmniInfer: System-Wide Acceleration Techniques for Optimizing LLM Serving Throughput and Latency
• Matrix: Peer-to-Peer Multi-Agent Synthetic Data Generation Framework
• PAT: Accelerating LLM Decoding via Prefix-Aware Attention with Resource Efficient Multi-Tile Kernel
• Guiding the Inner Eye: A Framework for Hierarchical and Flexible Visual Grounded Reasoning
• Statistical Independence Aware Caching for LLM Workflows
• IntAttention: A Fully Integer Attention Pipeline for Efficient Edge Inference
• Automated Dynamic AI Inference Scaling on HPC-Infrastructure: Integrating Kubernetes, Slurm and vLLM
• Thinking With Bounding Boxes: Enhancing Spatio-Temporal Video Grounding via Reinforcement Fine-Tuning
• Co-Training Vision Language Models for Remote Sensing Multi-task Learning
• Self-Guided Defense: Adaptive Safety Alignment for Reasoning Models via Synthesized Guidelines
• Orthographic Constraint Satisfaction and Human Difficulty Alignment in Large Language Models
• Reducing Latency of LLM Search Agent via Speculation-based Algorithm-System Co-Design
• Gated KalmaNet: A Fading Memory Layer Through Test-Time Ridge Regression
• DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
• Aragog: Just-in-Time Model Routing for Scalable Serving of Agentic Workflows
• Chatty-KG: A Multi-Agent AI System for On-Demand Conversational Question Answering over Knowledge Graphs
• Efficient Multi-Adapter LLM Serving via Cross-Model KV-Cache Reuse with Activated LoRA
• Reinforcing Action Policies by Prophesying
• DiFR: Inference Verification Despite Nondeterminism
• A note on the impossibility of conditional PAC-efficient reasoning in large language models
• MTBBench: A Multimodal Sequential Clinical Decision-Making Benchmark in Oncology
• Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios
• Beluga: A CXL-Based Memory Architecture for Scalable and Efficient LLM KVCache Management
• Inferix: A Block-Diffusion based Next-Generation Inference Engine for World Simulation
• Beyond Relational: Semantic-Aware Multi-Modal Analytics with LLM-Native Query Optimization
• Revisiting Feedback Models for HyDE
• Syn-GRPO: Self-Evolving Data Synthesis for MLLM Perception Reasoning
• LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling
• Latent Collaboration in Multi-Agent Systems
• SLMFix: Leveraging Small Language Models for Error Fixing with Reinforcement Learning
• ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models
• SWAN: Sparse Winnowed Attention for Reduced Inference Memory via Decompression-Free KV-Cache Compression
• Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models
• Periodic Asynchrony: An On-Policy Approach for Accelerating LLM Reinforcement Learning
• KernelBand: Boosting LLM-based Kernel Optimization with a Hierarchical and Hardware-aware Multi-armed Bandit
• Musical Score Understanding Benchmark: Evaluating Large Language Models' Comprehension of Complete Musical Scores
• VADE: Variance-Aware Dynamic Sampling via Online Sample-Level Difficulty Estimation for Multimodal RL
• Efficient Multi-Hop Question Answering over Knowledge Graphs via LLM Planning and Embedding-Guided Search
• Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost
• RAGPulse: An Open-Source RAG Workload Trace to Optimize RAG Serving Systems
• Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatch
• E³-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models
• Arctic-Extract Technical Report
• A Counterfactual LLM Framework for Detecting Human Biases: A Case Study of Sex/Gender in Emergency Triage
• Taming the Long-Tail: Efficient Reasoning RL Training with Adaptive Drafter
• Evolution Strategies at the Hyperscale
• On 10x Better Scalability: KV Stores Scale Up KV Cache
• Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning
• OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe
• AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization
• Mind the Motions: Benchmarking Theory-of-Mind in Everyday Body Language
• German General Social Survey Personas: A Survey-Derived Persona Prompt Collection for Population-Aligned LLM Studies
• Enhancing Reliability across Short and Long-Form QA via Reinforcement Learning
• EntroPIC: Towards Stable Long-Term Training of LLMs via Entropy Stabilization with Proportional-Integral Control
• Seer: Online Context Learning for Fast Synchronous LLM Reinforcement Learning
• LiteCache: A Query Similarity-Driven, GPU-Centric KVCache Subsystem for Efficient LLM Inference
• Agent-R1: Training Powerful LLM Agents with End-to-End Reinforcement Learning
• ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning
• NeuroPath: Neurobiology-Inspired Path Tracking and Reflection for Semantically Coherent Retrieval
• Beat the long tail: Distribution-Aware Speculative Decoding for RL Training
• Generalist Foundation Models Are Not Clinical Enough for Hospital Operations
• Reasoning Shapes Alignment: Investigating Cultural Alignment in Large Reasoning Models with Cultural Norms
• Dissecting and Re-architecting 3D NAND Flash PIM Arrays for Efficient Single-Batch Token Generation in LLMs
• Comparative Analysis of Large Language Model Inference Serving Systems: A Performance Study of vLLM and HuggingFace TGI
• Scaling Graph Chain-of-Thought Reasoning: A Multi-Agent Framework with Efficient LLM Serving
• TPS-Bench: Evaluating AI Agents' Tool Planning & Scheduling Abilities in Compounding Tasks
• Tailored Primitive Initialization is the Secret Key to Reinforcement Learning
• Optimal Self-Consistency for Efficient Reasoning with Large Language Models
• CrossVid: A Comprehensive Benchmark for Evaluating Cross-Video Reasoning in Multimodal Large Language Models
• Fusionista2.0: Efficiency Retrieval System for Large-Scale Datasets
• On the Entropy Calibration of Language Models
• KVSwap: Disk-aware KV Cache Offloading for Long-Context On-device Inference
• FarSkip-Collective: Unhobbling Blocking Communication in Mixture of Experts Models
• BhashaKritika: Building Synthetic Pretraining Data at Scale for Indic Languages
• VIDEOP2R: Video Understanding from Perception to Reasoning
• Generative Caching for Structurally Similar Prompts and Responses
• Improving LLM's Attachment to External Knowledge In Dialogue Generation Tasks Through Entity Anonymization
• Optimizing Mixture of Block Attention
• ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference
• Reasoning About Intent for Ambiguous Requests
• Efficient Thought Space Exploration Through Strategic Intervention
• Harli: SLO-Aware Co-location of LLM Inference and PEFT-based Finetuning on Model-as-a-Service Platforms
• SPAN: Benchmarking and Improving Cross-Calendar Temporal Reasoning of Large Language Models
• Lit Silicon: A Case Where Thermal Imbalance Couples Concurrent Execution in Multiple GPUs
• LLM Inference Beyond a Single Node: From Bottlenecks to Mitigations with Fast All-Reduce Communication
• Seer Self-Consistency: Advance Budget Estimation for Adaptive Test-Time Scaling
• Bench360: Benchmarking Local LLM Inference from 360°
• PrefixNLI: Detecting Factual Inconsistencies as Soon as They Arise
• Beyond Task-Oriented and Chitchat Dialogues: Proactive and Transition-Aware Conversational Agents
• Think-at-Hard: Selective Latent Iterations to Improve Reasoning Language Models
• The Path Not Taken: RLVR Provably Learns Off the Principals
• Understanding Inference Scaling for LLMs: Bottlenecks, Trade-offs, and Performance Principles
• Event Tensor: A Unified Abstraction for Compiling Dynamic Megakernel
• AlignSurvey: A Comprehensive Benchmark for Human Preferences Alignment in Social Surveys
• MURPHY: Multi-Turn GRPO for Self Correcting Code Generation
• Judging by the Rules: Compliance-Aligned Framework for Modern Slavery Statement Monitoring
• Streaming Tensor Program: A streaming abstraction for dynamic parallelism
• Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts
• When, What, and How: Rethinking Retrieval-Enhanced Speculative Decoding
• TCM-Eval: An Expert-Level Dynamic and Extensible Benchmark for Traditional Chinese Medicine
• Selecting Auxiliary Data via Neural Tangent Kernels for Low-Resource Domains
• More Agents Helps but Adversarial Robustness Gap Persists
• Argus: Quality-Aware High-Throughput Text-to-Image Inference Serving System
• Textual Self-attention Network: Test-Time Preference Optimization through Textual Gradient-based Attention
• Data Trajectory Alignment for LLM Domain Adaptation: A Two-Phase Synthesis Framework for Telecommunications Mathematics
• LLMServingSim2.0: A Unified Simulator for Heterogeneous Hardware and Serving Techniques in LLM Infrastructure
• You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
• Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning
• LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAs
• Tiny Model, Big Logic: Diversity-Driven Optimization Elicits Large-Model Reasoning Ability in VibeThinker-1.5B
• Optimizing Long-context LLM Serving via Fine-grained Sequence Parallelism
• Lethe: Layer- and Time-Adaptive KV Cache Pruning for Reasoning-Intensive LLM Serving
• Kunlun Anomaly Troubleshooter: Enabling Kernel-Level Anomaly Detection and Causal Reasoning for Large Model Distributed Inference
• CoEdge-RAG: Optimizing Hierarchical Scheduling for Retrieval-Augmented LLMs in Collaborative Edge Computing
• Can a Small Model Learn to Look Before It Leaps? Dynamic Learning and Proactive Correction for Hallucination Detection
• Hilbert-Guided Sparse Local Attention
• CoT-X: An Adaptive Framework for Cross-Model Chain-of-Thought Transfer and Optimization
• Learning from Online Videos at Inference Time for Computer-Use Agents
• The Future of Fully Homomorphic Encryption System: from a Storage I/O Perspective
• BudgetMem: Learning Selective Memory Policies for Cost-Efficient Long-Context Processing in Language Models
• LLMs as Packagers of HPC Software
• DuetServe: Harmonizing Prefill and Decode for LLM Serving via Adaptive GPU Multiplexing
• Memory- and Latency-Constrained Inference of Large Language Models via Adaptive Split Computing
• MIDI-LLM: Adapting Large Language Models for Text-to-MIDI Music Generation
• NVIDIA Nemotron Nano V2 VL
• SynQuE: Estimating Synthetic Dataset Quality Without Annotations
• Surprisal reveals diversity gaps in image captioning and different scorers change the story
• What's in Common? Multimodal Models Hallucinate When Reasoning Across Scenes
• From Prompts to Power: Measuring the Energy Footprint of LLM Inference
• AnchorTP: Resilient LLM Inference with State-Preserving Elastic Tensor Parallelism
• CareMedEval dataset: Evaluating Critical Appraisal and Reasoning in the Biomedical Field
• CoPRIS: Efficient and Stable Reinforcement Learning via Concurrency-Controlled Partial Rollout with Importance Sampling
• Accelerating Physical Property Reasoning for Augmented Visual Cognition
• SnapStream: Efficient Long Sequence Decoding on Dataflow Accelerators
• UTF-8 Plumbing: Byte-level Tokenizers Unavoidably Enable LLMs to Generate Ill-formed UTF-8
• ContextPilot: Fast Long-Context Inference via Context Reuse
• Divide, Cache, Conquer: Dichotomic Prompting for Efficient Multi-Label LLM-Based Classification
• In Good GRACEs: Principled Teacher Selection for Knowledge Distillation
• CostBench: Evaluating Multi-Turn Cost-Optimal Planning and Adaptation in Dynamic Environments for LLM Tool-Use Agents
• Verifying LLM Inference to Prevent Model Weight Exfiltration
• From Models to Operators: Rethinking Autoscaling Granularity for Large Generative Models
• Opportunistic Expert Activation: Batch-Aware Expert Routing for Faster Decode Without Retraining
• Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live
• Using Span Queries to Optimize for Cache and Attention Locality
• Can LLMs subtract numbers?
• Rethinking LLM Human Simulation: When a Graph is What You Need
• Flashlight: PyTorch Compiler Extensions to Accelerate Attention Variants
• Democratizing LLM Efficiency: From Hyperscale Optimizations to Universal Deployability
• KV Cache Transform Coding for Compact Storage in LLM Inference
• Knowledge Elicitation with Large Language Models for Interpretable Cancer Stage Identification from Pathology Reports
• IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation
• FlexiCache: Leveraging Temporal Stability of Attention Heads for Efficient KV Cache Management
• AGRAG: Advanced Graph-based Retrieval-Augmented Generation for LLMs
• FREESH: Fair, Resource- and Energy-Efficient Scheduling for LLM Serving on Heterogeneous GPUs
• Efficient Reinforcement Learning for Large Language Models with Intrinsic Exploration
• A CPU-Centric Perspective on Agentic AI
• Separate the Wheat from the Chaff: Winnowing Down Divergent Views in Retrieval Augmented Generation
• Leveraging Multi-Agent System (MAS) and Fine-Tuned Small Language Models (SLMs) for Automated Telecom Network Troubleshooting
• EPARA: Parallelizing Categorized AI Inference in Edge Clouds
• Tree Training: Accelerating Agentic LLMs Training via Shared Prefix Reuse
• ToM: Leveraging Tree-oriented MapReduce for Long-Context Reasoning in Large Language Models
• Sherlock: Reliable and Efficient Agentic Workflow Execution
• LingGym: How Far Are LLMs from Thinking Like Field Linguists?
• RDMA Point-to-Point Communication for LLM Systems
• EncouRAGe: Evaluating RAG Local, Fast, and Reliable
• VCORE: Variance-Controlled Optimization-based Reweighting for Chain-of-Thought Supervision
• DeepCompress: A Dual Reward Strategy for Dynamically Exploring and Compressing Reasoning Chains
• Glia: A Human-Inspired AI for Automated Systems Design and Optimization
• Unstructured Data Analysis using LLMs: A Comprehensive Benchmark
• SpecAttn: Speculating Sparse Attention
• Independent Clinical Evaluation of General-Purpose LLM Responses to Signals of Suicide Risk
• FlowMesh: A Service Fabric for Composable LLM Workflows
• Defeating the Training-Inference Mismatch via FP16
• Cross-Platform Evaluation of Reasoning Capabilities in Foundation Models
• Emu3.5: Native Multimodal Models are World Learners
• EdgeRunner 20B: Military Task Parity with GPT-5 while Running on the Edge
• Data-Efficient RLVR via Off-Policy Influence Guidance
• Counteracting Matthew Effect in Self-Improvement of LVLMs through Head-Tail Re-balancing
• BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuning
• The Geometry of Dialogue: Graphing Language Models to Reveal Synergistic Teams for Multi-Agent Collaboration
• Pragmatic Theories Enhance Understanding of Implied Meanings in LLMs
• Angular Steering: Behavior Control via Rotation in Activation Space
• One Model to Critique Them All: Rewarding Agentic Tool-Use via Efficient Reasoning
• Beyond Benchmarks: The Economics of AI Inference
• Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
• Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Serving
• Limits of Generalization in RLVR: Two Case Studies in Mathematical Reasoning
• PORTool: Tool-Use LLM Training with Rewarded Tree
• Roleplaying with Structure: Synthetic Therapist-Client Conversation Generation from Questionnaires
• PureKV: Plug-and-Play KV Cache Optimization with Spatial-Temporal Sparse Attention for Vision-Language Large Models
• Communication and Verification in LLM Agents towards Collaboration under Information Asymmetry
• Generalized Pseudo-Relevance Feedback
• NetEcho: From Real-World Streaming Side-Channels to Full LLM Conversation Recovery
• Serve Programs, Not Prompts
• Not ready for the bench: LLM legal interpretation is unstable and out of step with human judgments
• From Role Prompt to Infinite Thinking: Exploiting Persona Conditioning for Inference Cost Attacks in LLMs
• CARE: A Multimodal Corpus for Studying Speech and Non-Verbal Communication Across Multiple Medical Conditions
• Cache Merging as a Convergent Replicated State for Multi-Agent Latent Reasoning
• BM25 Wins at Scale: A Scaling Study of Retrieval-Augmented Generation Paradigms
• LLMET: Enabling Cross-Layer Evaluation of Emerging M3D Memories for Energy-Efficient LLM Serving
• StrataCL: Fabric-Native Communication Library for Production Supernodes
• NELSSA: A GPU-PNM Heterogeneous System for Mixed-Length LLM Serving via Length-based Request Placement
• ServerlessT2I: Efficient Text-to-Image Workflow Serving on a Serverless Platform
• From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs
• Steering Instruction Hierarchies at Inference Time
• GuideSkill: Evolving Executable LLM Agent Skills for Guideline-Grounded Clinical Reasoning
• Probing the Origins of Reasoning Performance: Representational Quality for Mathematical Problem-Solving in RL vs. SFT Fine-Tuned Models
• The Fabric Is the Cluster Driver: Cross-Layer eBPF Policies for GPU-CXL Fabrics
• RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention
• Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers
• RAPID: An Efficient Reinforcement Learning Algorithm for Small Language Models
• ElasticMoE: An Efficient Auto Scaling Method for Mixture-of-Experts Models
• AgentSnare: Learning to Delay, Divert, and Defuse Autonomous Penetration Agents
• Latent-IM: Latent Interaction Management for Speech LLMs
• Breaking the Ice: Analyzing Cold Start Latency in vLLM
• ThriftAttention: Selective Mixed Precision for Long-Context FP4 Attention
• SSV: Sparse Speculative Verification for Efficient LLM Inference
• The Consensus Trap: Rescuing Multi-Agent LLMs from Adversarial Majorities via Token-Level Collaboration
• The Stretto Execution Engine for LLM-Augmented Data Systems
• TrajSelector: Harnessing Latent Representations for Efficient and Effective Best-of-N in Large Reasoning Model
• Parallel Context-of-Experts Decoding for Retrieval Augmented Generation
• Beyond One-Size-Fits-All: Personalized Harmful Content Detection with In-Context Learning
• MoEntwine: Unleashing the Potential of Wafer-scale Chips for Large-scale Expert Parallel Inference
• Opt4GPTQ: Co-Optimizing Memory and Computation for 4-bit GPTQ Quantized LLM Inference on Heterogeneous Platforms
• SPICE: Self-Play In Corpus Environments Improves Reasoning
• Optimizing Retrieval for RAG via Reinforced Contrastive Learning
• Instant Personalized Large Language Model Adaptation via Hypernetwork
• Long-Context Modeling with Dynamic Hierarchical Sparse Attention for On-Device LLMs
• What Limits Agentic Systems Efficiency?
• Parallel Loop Transformer for Efficient Test-Time Computation Scaling
• APTBench: Benchmarking Agentic Potential of Base LLMs During Pre-Training
• zFLoRA: Zero-Latency Fused Low-Rank Adapters
• Lookahead Tree-Based Rollouts for Enhanced Trajectory-Level Exploration in Reinforcement Learning with Verifiable Rewards
• MCP-Flow: Facilitating LLM Agents to Master Real-World, Diverse and Scaling MCP Tools
• Reinforcement Learning for Long-Horizon Multi-Turn Search Agents
• Beyond Neural Incompatibility: Easing Cross-Scale Knowledge Transfer in Large Language Models through Latent Semantic Alignment
• Beyond Line-Level Filtering for the Pretraining Corpora of LLMs
• Pie: A Programmable Serving System for Emerging LLM Applications
• ChessQA: Evaluating Large Language Models for Chess Understanding
• Fortytwo: Swarm Inference with Peer-Ranked Consensus
• Compiler.next: A Search-Based Compiler to Power the AI-Native Future of Software Engineering
• From Benchmarks to Business Impact: Deploying IBM Generalist Agent in Enterprise Production
• Beyond Understanding: Evaluating the Pragmatic Gap in LLMs' Cultural Processing of Figurative Language
• Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets
• IPQA: A Benchmark for Core Intent Identification in Personalized Question Answering
• VOLD: Reasoning Transfer from LLMs to Vision-Language Models via On-Policy Distillation
• Learning to Reason Efficiently with Discounted Reinforcement Learning
• MR-Align: Meta-Reasoning Informed Factuality Alignment for Large Reasoning Models
• Fast-MIA: Efficient and Scalable Membership Inference for LLMs
• Batch Speculative Decoding Done Right
• VEHME: A Vision-Language Model For Evaluating Handwritten Mathematics Expressions
• Iterative Layer Pruning for Efficient Translation Inference
• LooGLE v2: Are LLMs Ready for Real World Long Dependency Challenges?
• E2Rank: Your Text Embedding can Also be an Effective and Efficient Listwise Reranker
• Measure what Matters: Psychometric Evaluation of AI with Situational Judgment Tests
• LightAgent: Mobile Agentic Foundation Models
• A Multimodal Benchmark for Framing of Oil & Gas Advertising and Potential Greenwashing Detection
• A Data-Centric Approach to Multilingual E-Commerce Product Search: Case Study on Query-Category and Query-Item Relevance
• Boosting Accuracy and Efficiency of Budget Forcing in LLMs via Reinforcement Learning for Mathematical Reasoning
• Excision Score: Evaluating Edits with Surgical Precision
• Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generation
• Customizing Open Source LLMs for Quantitative Medication Attribute Extraction across Heterogeneous EHR Systems
• Stateful KV Cache Management for LLMs: Balancing Space, Time, Accuracy, and Positional Fidelity
• Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset
• GRATING: Low-Latency and Memory-Efficient Semantic Selection on Device
• FreeChunker: A Cross-Granularity Chunking Framework
• PartNeXt: A Next-Generation Dataset for Fine-Grained and Hierarchical 3D Part Understanding
• Rank-GRPO: Training LLM-based Conversational Recommender Systems with Reinforcement Learning
• AI PB: A Grounded Generative Agent for Personalized Investment Insights
• BioCAP: Exploiting Synthetic Captions Beyond Labels in Biological Foundation Models
• Limits of PRM-Guided Tree Search for Mathematical Reasoning with LLMs
• RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging
• CoSense-LLM: Semantics at the Edge with Cost- and Uncertainty-Aware Cloud-Edge Cooperation
• LLavaCode: Compressed Code Representations for Retrieval-Augmented Code Generation
• Detecting Latin in Historical Books with Large Language Models: A Multimodal Benchmark
• MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
• From Large to Small: Transferring CUDA Optimization Expertise via Reasoning Graph
• Unified Reinforcement and Imitation Learning for Vision-Language Models
• Structured and Abstractive Reasoning on Multi-modal Relational Knowledge Images
• AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders
• DiffAdapt: Difficulty-Adaptive Reasoning for Token-Efficient LLM Inference
• SecureInfer: Heterogeneous TEE-GPU Architecture for Privacy-Critical Tensors for Large Language Model Deployment
• RLBoost: Harvesting Preemptible Resources for Cost-Efficient Reinforcement Learning on LLMs
• DiSRouter: Distributed Self-Routing for LLM Selections
• Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Model
• Reasoning Language Model Inference Serving Unveiled: An Empirical Study
• Tokencake: A KV-Cache-centric Serving Framework for LLM-based Multi-Agent Applications
• WebDevJudge: Evaluating (M)LLMs as Critiques for Web Development Quality
• EfficientNav: Towards On-Device Object-Goal Navigation with Navigation Map Caching and Retrieval
• SLICE: SLO-Driven Scheduling for LLM Inference on Edge Computing Devices
• From Retrieval to Generation: Unifying External and Parametric Knowledge for Medical Question Answering
• EdgeReasoning: Characterizing Reasoning LLM Deployment on Edge GPUs
• Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs
• RESCUE: Retrieval Augmented Secure Code Generation
• MTraining: Distributed Dynamic Sparse Attention for Efficient Ultra-Long Context Training
• PoSh: Using Scene Graphs To Guide LLMs-as-a-Judge For Detailed Image Descriptions
• How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practices
• Zero‐ and few‐shot prompting of generative large language models provides weak assessment of risk of bias in clinical trials
• ReXMoE: Reusing Experts with Minimal Overhead in Mixture-of-Experts
• Disparities in Multilingual LLM-Based Healthcare Q&A
• Mamba4Net: Distilled Hybrid Mamba Large Language Models For Networking
• Investigating Thinking Behaviours of Reasoning-Based Language Models for Social Bias Mitigation
• Prompt-MII: Meta-Learning Instruction Induction for LLMs
• DeepAnalyze: Agentic Large Language Models for Autonomous Data Science
• Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedback
• Mixed-Precision Quantization for Language Models: Techniques and Prospects
• Accelerating Mobile Language Model via Speculative Decoding and NPU-Coordinated Execution
• MARSHAL: Incentivizing Multi-Agent Reasoning via Self-Play with Strategic LLMs
• Enhancing reliability in AI inference services: An empirical study on real production incidents
• Synera: Synergistic LLM Serving across Device and Cloud at Scale
• Adaptive Minds: Empowering Agents with LoRA-as-Tools
• Tail-Optimized Caching for LLM Inference
• Predicting Task Performance with Context-aware Scaling Laws
• Speculative Model Risk in Healthcare AI: Using Storytelling to Surface Unintended Harms
• xLLM Technical Report
• ScalePool: Hybrid XLink-CXL Fabric for Composable Resource Disaggregation in Unified Scale-up Domains
• MX+: Pushing the Limits of Microscaling Formats for Efficient Large Language Model Serving
• ToolTweak: An Attack on Tool Selection in LLM-based Agents
• Budget-aware Test-time Scaling via Discriminative Verification
• Cortex: Workflow-Aware Resource Pooling and Scheduling for Agentic Serving
• Efficiently Executing High-throughput Lightweight LLM Inference Applications on Heterogeneous Opportunistic GPU Clusters with Pervasive Context Management
• Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Math
• FIRST: Federated Inference Resource Scheduling Toolkit for Scientific AI Model Access
• Adaptive Rescheduling in Prefill-Decode Disaggregated LLM Inference
• NOSA: Native and Offloadable Sparse Attention
• Deflanderization for Game Dialogue: Balancing Character Authenticity with Task Execution in LLM-based NPCs
• M²PO: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation
• F-BFQ: Flexible Block Floating-Point Quantization Accelerator for LLMs
• Protect: Towards Robust Guardrailing Stack for Trustworthy Enterprise LLM Systems
• BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
• Confidence-Based Response Abstinence: Improving LLM Trustworthiness via Activation-Based Uncertainty Estimation
• BoN Appetit Team at LeWiDi-2025: Best-of-N Test-time Scaling Can Not Stomach Annotation Disagreements (Yet)
• Development and Benchmarking of a Blended Human-AI Qualitative Research Assistant
• KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems
• Laminar: A Scalable Asynchronous RL Post-Training Framework
• A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthiness
• Locket: Robust Feature-Locking Technique for Language Models
• FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
• LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokens
• Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models
• Representation-Based Exploration for Language Models: From Test-Time to Post-Training
• Enhancing Long Chain-of-Thought Reasoning through Multi-Path Plan Aggregation
• From to : Multidimensional Supervision of Reasoning Process for LLM Optimization
• Part II: ROLL Flash -- Accelerating RLVR and Agentic Training with Asynchrony
• Collaborative Shadows: Distributed Backdoor Attacks in LLM-Based Multi-Agent Systems
• XQuant: Achieving Ultra-Low Bit KV Cache Quantization with Cross-Layer Compression
• An Explorative Study on Distributed Computing Techniques in Training and Inference of Large Language Models
• How²: How to learn from procedural How-to questions
• From Reasoning LLMs to BERT: A Two-Stage Distillation Framework for Search Relevance
• LogiNumSynth: Synthesizing Joint Logical-Numerical Reasoning Problems for Language Models
• Parallel Scaling Law: Unveiling Reasoning Generalization through A Cross-Linguistic Perspective
• The Social Cost of Intelligence: Emergence, Propagation, and Amplification of Stereotypical Bias in Multi-Agent Systems
• LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblance
• Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
• KVComm: Enabling Efficient LLM Communication through Selective KV Sharing
• RePro: Training Language Models to Faithfully Recycle the Web for Pretraining
• DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism
• Integrating Large Language Models and Reinforcement Learning for Sentiment-Driven Quantitative Trading
• SASER: Stego attacks on open-source LLMs
• Traj-CoA: Patient Trajectory Modeling via Chain-of-Agents for Lung Cancer Risk Prediction
• RECON: Reasoning with Condensation for Efficient Retrieval-Augmented Generation
• Clarifying Semantics of In-Context Examples for Unit Test Generation
• LLM-Friendly Knowledge Representation for Customer Support
• DynaSpec: Context-aware Dynamic Speculative Sampling for Large-Vocabulary Language Models
• Grounded AI for Code Review: Resource-Efficient Large-Model Serving in Enterprise Pipelines
• CacheClip: Accelerating RAG with Effective KV Cache Reuse
• Think Twice to See More: Iterative Visual Reasoning in Medical VLMs
• MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inference
• Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
• AutoMegaKernel: A Statically-Checked Agent Harness for Self-Retargeting Megakernel Synthesis
• AURA: Action-Gated Memory for Robot Policies at Constant VRAM
• Attention Once Is All You Need: Efficient Streaming Inference with Stateful Transformers
• CCL-Bench 1.0: A Trace-Based Benchmark for LLM Infrastructure
• OntoLogX: Ontology-Guided Knowledge Graph Extraction from Cybersecurity Logs with Large Language Models
• Accelerating Attention with Basis Decomposition
• Faster LLM Inference via Sequential Monte Carlo
• Demystifying the Silence of Correctness Bugs in PyTorch Compiler
• Therefore I am. I Think
• NCCL EP: Towards a Unified Expert Parallel Communication API for NCCL
• DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference
• Private LLM Inference on Consumer Blackwell GPUs: A Practical Guide for Cost-Effective Local Deployment in SMEs
• DELTA: Dynamic Layer-Aware Token Attention for Efficient Long-Context Reasoning
• Token Is All You Price
• Doc2Query++: Topic-Coverage based Document Expansion and its Application to Dense Retrieval via Dual-Index Fusion
• DICE: Structured Reasoning in LLMs through SLM-Guided Chain-of-Thought Correction
• CLARity: Reasoning Consistency Alone Can Teach Reinforced Experts
• Inflated Excellence or True Performance? Rethinking Medical Diagnostic Benchmarks with Dynamic Evaluation
• TripScore: Benchmarking and rewarding real-world travel planning with fine-grained evaluation
• MIRAGE: Runtime Scheduling for Multi-Vector Image Retrieval with Hierarchical Decomposition
• CapGeo: A Caption-Assisted Approach to Geometric Reasoning
• On the Provable Performance Guarantee of Efficient Reasoning Models
• ProxRouter: Proximity-Weighted LLM Query Routing for Improved Robustness to Outliers
• Which Heads Matter for Reasoning? RL-Guided KV Cache Compression
• DeepPrune: Parallel Scaling without Inter-trace Redundancy
• The Alignment Waltz: Jointly Training Agents to Collaborate for Safety
• Fewer Weights, More Problems: A Practical Attack on LLM Pruning
• OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment
• Self-Improving LLM Agents at Test-Time
• SPAD: Specialized Prefill and Decode Hardware for Disaggregated LLM Inference
• From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill
• When to Reason: Semantic Router for vLLM
• Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation
• Reasoning for Hierarchical Text Classification: The Case of Patents
• Opt-ICL at LeWiDi-2025: Maximizing In-Context Signal from Rater Examples via Meta-Learning
• VRPAgent: LLM-Driven Discovery of Heuristic Operators for Vehicle Routing Problems
• OpenJAI-v1.0: An Open Thai Large Language Model
• Experiential Reinforcement Learning
• Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Models
• TetriServe: Efficiently Serving Mixed DiT Workloads
• Executable Counterfactuals: Improving LLMs' Causal Reasoning Through Code
• Study on LLMs for Promptagator-Style Dense Retriever Training
• XRPO: Pushing the limits of GRPO with Targeted Exploration and Exploitation
• Rethinking KL Regularization in RLHF: From Value Estimation to Gradient Optimization
• On the Role of Temperature Sampling in Test-Time Scaling
• What MLLMs Learn about When they Learn about Multimodal Reasoning
• LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
• POME: Post Optimization Model Edit via Muon-style Projection
• OWL: Overcoming Window Length-Dependence in Speculative Decoding for Long-Context Inputs
• Expanding the Action Space of LLMs to Reason Beyond Language
• GAZE:Governance-Aware pre-annotation for Zero-shot World Model Environments
• FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering
• Instructional Goal-Aligned Question Generation for Student Evaluation in Virtual Lab Settings: How Closely Do LLMs Actually Align?
• Training Dynamics Impact Post-Training Quantization Robustness
• VecInfer: Efficient LLM Inference with Low-Bit KV Cache via Outlier-Suppressed Vector Quantization
• SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation
• RoSE: Round-robin Synthetic Data Evaluation for Selecting LLM Generators without Human Test Sets
• CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credits
• lm-Meter: Unveiling Runtime Inference Latency for On-Device Language Models
• ARISE: An Adaptive Resolution-Aware Metric for Test-Time Scaling Evaluation in Large Reasoning Models
• The Anatomy of a Triton Attention Kernel
• Evaluating the Sensitivity of LLMs to Harmful Contents in Long Input
• On the Role of Difficult Prompts in Self-Play Preference Optimization
• MixReasoning: Switching Modes to Think
• When Importance Sampling Misallocates Credit: Asymmetric Ratios for Outcome-Supervised RL
• Staircase Streaming for Low-Latency Multi-Agent Inference
• Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages
• Stratum: System-Hardware Co-Design with Tiered Monolithic 3D-Stackable DRAM for Efficient MoE Serving
• Reinforcement Learning for Clinical Reasoning: Aligning LLMs with ACR Imaging Appropriateness Criteria
• Exploring OCR-augmented Generation for Bilingual VQA
• Transformers Discover Molecular Structure Without Graph Priors
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with PsiloQA
• Multilingual Routing in Mixture-of-Experts
• AlphaApollo: Orchestrating Foundation Models and Professional Tools into a Self-Evolving System for Deep Agentic Reasoning
• Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought
• PatternKV: Flattening KV Representation Expands Quantization Headroom
• Selective Expert Guidance for Effective and Diverse Exploration in Reinforcement Learning of LLMs
• SPOGW: a Score-based Preference Optimization method via Group-Wise comparison for workflows
• Slow-Fast Policy Optimization: Reposition-Before-Update for LLM Reasoning
• Toward a unified framework for data-efficient evaluation of large language models
• Distilling Reasoning into Student LLMs: Local Naturalness for Selecting Teacher Data
• Evaluation of Clinical Trials Reporting Quality using Large Language Models
• Doctor-R1: Mastering Clinical Inquiry with Experiential Agentic Reinforcement Learning
• Exact Causal Attention with 10% Fewer Operations
• An Early Exploration of Deep-Learning-Driven Prefetching for Far Memory
• AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework
• Locate-Then-Examine: Grounded Region Reasoning Improves Detection of AI-Generated Images
• Don't Pass@k: A Bayesian Framework for Large Language Model Evaluation
• SATER: A Self-Aware and Token-Efficient Approach to Routing and Cascading
• Round-trip Reinforcement Learning: Self-Consistent Training for Better Chemical LLMs
• AudioToolAgent: An Agentic Framework for Audio-Language Models
• TokenFlow: Responsive LLM Text Streaming Serving under Request Burst via Preemptive Scheduling
• GRAD: Generative Retrieval-Aligned Demonstration Sampler for Efficient Few-Shot Reasoning
• Prosperity before Collapse: How Far Can Off-Policy RL Reach with Stale Data on LLMs?
• mR3: Multilingual Rubric-Agnostic Reward Reasoning Models
• GEM: A Gym for Agentic LLMs
• QUASAR: Quantum Assembly Code Generation Using Tool-Augmented LLMs via Agentic RL
• When Silence Matters: The Impact of Irrelevant Audio on Text Reasoning in Large Audio-Language Models
• Rethinking Reward Models for Multi-Domain Test-Time Scaling
• Automated Structured Radiology Report Generation with Rich Clinical Context
• In-Place Feedback: Reliable Refinement for Multi-Turn Expert-LLM Collaboration
• Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Information
• Prompt Curriculum Learning for Efficient LLM Post-Training
• ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Models
• HiSpec: Hierarchical Speculative Decoding for LLMs
• On The Fragility of Benchmark Contamination Detection in Reasoning Models
• BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responses
• AgentFlux: Decoupled Fine-Tuning & Inference for On-Device Agentic Systems
• Attention as a Compass: Efficient Exploration for Process-Supervised RL in Reasoning Models
• Recursive Self-Aggregation Unlocks Deep Thinking in Large Language Models
• DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressively
• Rearchitecting Datacenter Lifecycle for AI: A TCO-Driven Framework
• Entropy After ⟨ /Think ⟩ for reasoning model early exiting
• SCUBA: Salesforce Computer Use Benchmark
• Efficient and Transferable Agentic Knowledge Graph RAG via Reinforcement Learning
• Thinking-Free Policy Initialization Makes Distilled Reasoning Models More Effective and Efficient Reasoners
• Parallax: Efficient LLM Inference Service over Decentralized Environment
• Human-MME: A Holistic Evaluation Benchmark for Human-Centric Multimodal Large Language Models
• Vocabulary Customization for Efficient Domain-Specific LLM Deployment
• CoLLM-NAS: Collaborative Large Language Models for Efficient Knowledge-Guided Neural Architecture Search
• RAGferee: Building Contextual Reward Models for Retrieval-Augmented Generation
• Accelerating LLM Inference with Precomputed Query Storage
• PerQ: Efficient Evaluation of Multilingual Text Personalization Quality
• SAIL: SRAM-Accelerated LLM Inference System with Lookup-Table-based GEMV
• Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocation
• Overthinking Reduction with Decoupled Rewards and Curriculum Data Scheduling
• OPPO: Accelerating PPO-based RLHF via Pipeline Overlap
• TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning
• Think Less, Label Better: Multi-Stage Domain-Grounded Synthetic Data Generation for Fine-Tuning Large Language Models in Telecommunications
• Nudging the Boundaries of LLM Reasoning
• LLM-Based Multi-Agent Blackboard System for Information Discovery in Data Science
• RouterArena: An Open Platform for Comprehensive Comparison of LLM Routers
• Hybrid Reward Normalization for Process-supervised Non-verifiable Agentic Tasks
• RADAR: Reasoning-Ability and Difficulty-Aware Routing for Reasoning LLMs
• EasySteer: A Unified Framework for High-Performance and Extensible LLM Steering
• Scaling Synthetic Task Generation for Agents via Exploration
• GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference
• CLPO: Curriculum Learning meets Policy Optimization for LLM Reasoning
• Random Policy Valuation is Enough for LLM Reasoning with Verifiable Rewards
• Intra-request branch orchestration for efficient LLM reasoning
• When Greedy Wins: Emergent Exploitation Bias in Meta-Bandit LLM Training
• SemShareKV: Efficient KVCache Sharing for Semantically Similar Prompts via Token-Level LSH Matching
• Fidel-TS: A High-Fidelity Benchmark for Multimodal Time Series Forecasting
• SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
• AdaThink-Med: Medical Adaptive Thinking with Uncertainty-Guided Length Calibration
• GSPR: Aligning LLM Safeguards as Generalizable Safety Policy Reasoners
• Towards Safe Reasoning in Large Reasoning Models via Corrective Intervention
• RServe: Overlapping Encoding and Prefill for Efficient LMM Inference
• Risk-Sensitive RL for Alleviating Exploration Dilemmas in Large Language Models
• SpecExit: Accelerating Large Reasoning Model via Speculative Exit
• ChessArena: A Chess Testbed for Evaluating Strategic Reasoning Capabilities of Large Language Models
• VIVA+: Human-Centered Situational Decision-Making
• From Score Distributions to Balance: Plug-and-Play Mixture-of-Experts Routing
• Unlocking Zero-Shot Geospatial Reasoning via Indirect Rewards
• Reasoning or Retrieval? A Study of Answer Attribution on Large Reasoning Models
• PIPer: On-Device Environment Setup via Online Reinforcement Learning
• MACE: A Hybrid LLM Serving System with Colocated SLO-aware Continuous Retraining Alignment
• Zero-Waiting Load Balancing with Heterogeneous Servers in Heavy Traffic
• Beyond the Exploration-Exploitation Trade-off: A Hidden State Approach for LLM Reasoning in RLVR
• GUI-Shepherd: Reliable Process Reward and Verification for Long-Sequence GUI Tasks
• SafeSearch: Automated Red-Teaming for the Safety of LLM-Based Search Agents
• Evaluating Program Semantics Reasoning with Type Inference in System F
• PreScope: Unleashing the Power of Prefetching for Resource-Constrained MoE Inference
• Pretraining Scaling Laws for Generative Evaluations of Language Models
• CORRECT: COndensed eRror RECognition via knowledge Transfer in multi-agent systems
• Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation
• Mapping Overlaps in Benchmarks through Perplexity in the Wild
• A Predictive and Synergistic Two-Layer Scheduling Framework for LLM Serving
• DentVLM: A Multimodal Vision-Language Model for Comprehensive Dental Diagnosis and Enhanced Clinical Practice
• SPEC-RL: Accelerating On-Policy Reinforcement Learning via Speculative Rollouts
• Leave No Observation Behind: Real-time Correction for VLA Action Chunks
• Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization
• Hilbert: Recursively Building Formal Proofs with Informal Reasoning
• EditGRPO: Reinforcement Learning with Post-Rollout Edits for Clinically Accurate Chest X-Ray Report Generation
• Language Models Can Learn from Verbal Feedback Without Scalar Rewards
• Variational Reasoning for Language Models
• Beyond statistical significance: Quantifying uncertainty and statistical variability in multilingual and multitask NLP evaluation
• Learn the Ropes, Then Trust the Wins: Self-imitation with Progressive Exploration for Agentic Reinforcement Learning
• Dynamic Experts Search: Enhancing Reasoning in Mixture-of-Experts LLMs at Test Time
• Boosting Pointer Analysis With LLM-Enhanced Allocation Function Detection
• Estimating the Empowerment of Language Model Agents
• FormalML: A Benchmark for Evaluating Formal Subgoal Completion in Machine Learning Theory
• InfiMed-Foundation: Pioneering Advanced Multimodal Medical Models with Compute-Efficient Pre-Training and Multi-Stage Fine-Tuning
• MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing
• SK2Decompile: LLM-based Two-Phase Binary Decompilation from Skeleton to Skin
• S2J: Bridging the Gap Between Solving and Judging Ability in Generative Reward Models
• ERGO: Efficient High-Resolution Visual Understanding for Vision-Language Models
• RISK: A Framework for GUI Agents in E-commerce Risk Management
• Think Smart, Not Hard: Difficulty Adaptive Reasoning for Large Audio Language Models
• AgentPack: A Dataset of Code Changes, Co-Authored by Agents and Humans
• Mixture-of-Visual-Thoughts: Exploring Context-Adaptive Reasoning Mode Selection for General Visual Reasoning
• RobustFlow: Towards Robust Agentic Workflow Generation
• Training-Free Multimodal Deepfake Detection via Graph Reasoning
• Think-on-Graph 3.0: Efficient and Adaptive LLM Reasoning on Heterogeneous Graphs via Multi-Agent Dual-Evolving Context Retrieval
• The Bias is in the Details: An Assessment of Cognitive Bias in LLMs
• Semantic F1 Scores: Fair Evaluation Under Fuzzy Class Boundaries
• Learning GUI Grounding with Spatial Reasoning from Visual Feedback
• VideoJudge: Bootstrapping Enables Scalable Supervision of MLLM-as-a-Judge for Video Understanding
• Nova: Real-Time Agentic Vision-Language Model Serving with Adaptive Cross-Stage Parallelization
• VC-Agent: An Interactive Agent for Customized Video Dataset Collection
• MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources
• Un-Doubling Diffusion: LLM-guided Disambiguation of Homonym Duplication
• Toward Robust and Efficient ML-Based GPU Caching for Modern Inference
• Prompt-Aware Scheduling for Low-Latency LLM Serving
• Generative AI for FFRDCs
• TyphoonMLA: A Mixed Naive-Absorb MLA Kernel For Shared Prefix
• RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Training
• UniSS: Unified Expressive Speech-to-Speech Translation with Your Voice
• When Instructions Multiply: Measuring and Estimating LLM Capabilities of Multiple Instructions Following
• Even More Kawaii than Real-Person-Driven VTubers? Understanding How Viewers Perceive AI-Driven VTubers
• ScaleDiff: Scaling Difficult Problems for Advanced Mathematical Reasoning
• Experience Deploying Containerized GenAI Services at an HPC Center
• Automated Multi-Agent Workflows for RTL Design
• Causal Understanding by LLMs: The Role of Uncertainty
• FastEagle: Cascaded Drafting for Accelerating Speculative Decoding
• Future Policy Aware Preference Learning for Mathematical Reasoning
• TRUEBench: Can LLM Response Meet Real-world Constraints as Productivity Assistant?
• SINAI at eRisk@CLEF 2025: Transformer-Based and Conversational Strategies for Depression Detection
• Gyges: Dynamic Cross-Instance Parallelism Transformation for Efficient LLM Inference
• Calibrated Reasoning: An Explanatory Verifier for Dynamic and Efficient Problem-Solving
• Embedding Domain Knowledge for Large Language Models via Reinforcement Learning from Augmented Generation
• Play by the Type Rules: Inferring Constraints for LLM Functions in Declarative Programs
• Are We Scaling the Right Thing? A System Perspective on Test-Time Scaling
• Advancing Speech Summarization in Multi-modal LLMs with Reinforcement Learning
• Look as You Leap: Planning Simultaneous Motion and Perception for High-DOF Robots
• Back from the Future: Key-Value Cache Management by Counter-Causal Surprise
• EMBL AI Librarian: Life-Sciences Knowledge Layer for AI Agents
• SemPIC: Learning Semantic Position-Independent KV Caches
• Echoverse: Deep, Evolving Environments for Training Computer-Use Agents at Scale
• WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
• Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Game
• SmartGen: Seamless Disaggregated LLM Inference with Selective KV Cache Transfer
• Cocktail-Talker: Multi-Speaker Dialog Modeling in Noisy Social Environments with Turn Action GRPO
• A Sparse Glimpse of the Whole: Train-Free Self-Speculative Decoding
• A Policy-Driven Runtime Layer for Agentic LLM Serving
• Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold
• Dimensionality and Measurement Precision in HLE's Multiple-Choice Subset
• Selecting Open-Weight Language Models for Zero-Shot Intent Classification: A Systematic Evaluation of 41 Models
• Clairvoyant: Predictive Shortest-Job-First Admission for Serial LLM Inference
• KernelGenBench: A Multi-Source and Multi-Chip Benchmark for LLM-based Kernel Generation
• SuperThoughts: Reasoning Tokens in Superposition
• Benchmarking Open-Ended Multi-Agent Coordination in Language Agents
• Context Recycling for Long-Horizon LLM Inference
• QCalEval: Benchmarking Vision-Language Models for Quantum Calibration Plot Understanding
• How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data
• RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Design
• DFlash: Block Diffusion for Flash Speculative Decoding
• GORGO: Online Tuning for Cross-Region Network-Aware LLM Serving
• ILRe: Intermediate Layer Retrieval for Context Compression in Causal Language Models
• AtomRAG: atomic query decomposition for multi-hop retrieval-augmented generation
• Scalable Engine and the Performance of Different LLM Models in a SLURM based HPC architecture
• CompLLM: Compression for Long Context Q&A
• YAC: Bridging Natural Language and Interactive Visual Exploration with Generative AI for Biomedical Data Discovery
• Soft Tokens, Hard Truths
• PipelineRL: Faster On-policy Reinforcement Learning for Long Sequence Generation
• APRIL: Active Partial Rollouts in Reinforcement Learning to Tame Long-tail Generation
• The Ranking Blind Spot: Decision Hijacking in LLM-based Text Ranking
• Confidential LLM Inference: Performance and Cost Across CPU and GPU TEEs
• Introducing LongCat-Flash-Thinking: A Technical Report
• Investigating Test-Time Scaling with Reranking for Machine Translation
• Speculate Deep and Accurate: Lossless and Training-Free Acceleration for Offloaded LLMs via Substitute Speculative Decoding
• A Multimodal Conversational Assistant for the Characterization of Agricultural Plots from Geospatial Open Data
• Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs
• Correlation or Causation: Analyzing the Causal Structures of LLM and LRM Reasoning Process
• Cronus: Efficient LLM inference on Heterogeneous GPU Clusters via Partially Disaggregated Prefill
• Exploiting Tree Structure for Credit Assignment in RL Training of LLMs
• Everyday Physics in Korean Contexts: A Culturally Grounded Physical Reasoning Benchmark
• Expert-as-a-Service: Towards Efficient, Scalable, and Robust Large-scale MoE Serving
• MapCoder-Lite: Distilling Multi-Agent Coding into a Single Small LLM
• Scaling, Simplification, and Adaptation: Lessons from Pretraining on Machine-Translated Text
• Cortex: Achieving Low-Latency, Cost-Efficient Remote Data Access For LLM via Semantic-Aware Knowledge Caching
• CFDA & CLIP at TREC iKAT 2025: Enhancing Personalized Conversational Search via Query Reformulation and Rank Fusion
• Qwen3-Omni Technical Report
• ExpertWeave: Efficiently Serving Expert-Specialized Fine-Tuned Adapters at Scale
• AirQA: A Comprehensive QA Dataset for AI Research with Instance-Level Evaluation
• ShadowServe: Interference-Free KV Cache Fetching for Distributed Prefix Caching
• Multi-level Diagnosis and Evaluation for Robust Tabular Feature Engineering with Large Language Models
• From Uniform to Heterogeneous: Tailoring Policy Optimization to Every Token's Nature
• SCAN: Self-Denoising Monte Carlo Annotation for Robust Process Reward Learning
• Advancing Reference-free Evaluation of Video Captions with Factual Analysis
• Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputs
• Rethinking the Role of Text Complexity in Language Model Pretraining
• Shift Parallelism: Low-Latency, High-Throughput LLM Inference for Dynamic Workloads
• Randomized Smoothing Meets Vision-Language Models
• TradingGroup: A Multi-Agent Trading System with Self-Reflection and Data-Synthesis
• SightSound-R1: Cross-Modal Reasoning Distillation from Vision to Audio Language Models
• ChartMaster: Advancing Chart-to-Code Generation with Real-World Charts and Chart Similarity Reinforcement Learning
• Relevance to Utility: Process-Supervised Rewrite for RAG
• GPO: Learning from Critical Steps to Improve LLM Reasoning
• Overhearing LLM Agents: A Survey, Taxonomy, and Roadmap
• SERVAL: Surprisingly Effective Zero-Shot Visual Document Retrieval Powered by Large Vision and Language Models
• LLM-OREF: An Open Relation Extraction Framework Based on Large Language Models
• MoE-Inference-Bench: Performance Evaluation of Mixture of Expert Large Language and Vision Models
• From Ground Trust to Truth: Disparities in Offensive Language Judgments on Contemporary Korean Political Discourse
• SALT4Decompile: Inferring Source-level Abstract Logic Tree for LLM-Based Binary Decompilation
• TreePO: Bridging the Gap of Policy Optimization and Efficacy and Inference Efficiency with Heuristic Tree-based Modeling
• ATTS: Asynchronous Test-Time Scaling via Conformal Prediction
• Adversarial Distilled Retrieval-Augmented Guarding Model for Online Malicious Intent Detection
• Synthetic bootstrapped pretraining
• DashboardQA: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboards
• Mimicking the Physicist's Eye:A VLM-centric Approach for Physics Formula Discovery
• PiERN: Token-Level Routing for Integrating High-Precision Computation and Reasoning
• Latent Traits and Cross-Task Transfer: Deconstructing Dataset Interactions in LLM Fine-tuning
• Hala Technical Report: Building Arabic-Centric Instruction & Translation Models at Scale
• CodeLSI: Leveraging Foundation Models for Automated Code Generation with Low-Rank Optimization and Domain-Specific Instruction Tuning
• MARS2 2025 Challenge on Multimodal Reasoning: Datasets, Methods, Results, Discussion, and Outlook
• Towards General Agentic Intelligence via Environment Scaling
• LATTS: Locally Adaptive Test-Time Scaling
• Scaling Up Throughput-oriented LLM Inference Applications on Heterogeneous Opportunistic GPU Clusters with Pervasive Context Management
• All Roads Lead to Rome: Graph-Based Confidence Estimation for Large Language Model Reasoning
• LLMs Can't Handle Peer Pressure: Crumbling under Multi-Agent Social Interactions
• ScaleDoc: Scaling LLM-based Predicates over Large Document Collections
• Gender-Neutral Rewriting in Italian: Models, Approaches, and Trade-offs
• AI Factories: It's time to rethink the Cloud-HPC divide
• TokenLake: A Unified Segment-level Prefix Cache Pool for Fine-grained Elastic Long-Context LLM Serving
• Reinforcement Learning enhanced Online Adaptive Clinical Decision Support via Digital Twin powered Policy and Treatment Effect optimized Reward
• The Better You Learn, The Smarter You Prune: Towards Efficient Vision-language-action Models via Differentiable Token Pruning
• BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokens
• Graph-R1: Incentivizing the Zero-Shot Graph Learning Capability in LLMs via Explicit Reasoning
• FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Prediction
• Biomedical Hypothesis Explainability with Graph-Based Context Retrieval
• Framing AI System Benchmarking as a Learning Task: FlexBench and the Open MLPerf Dataset
• Your Compiler is Backdooring Your Model: Understanding and Exploiting Compilation Inconsistency Vulnerabilities in Deep Learning Compilers
• Improving Table Understanding with LLMs and Entity-Oriented Search
• OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft
• VARCO-VISION-2.0 Technical Report
• Established Psychometric vs. Ecologically Valid Questionnaires: Rethinking Psychological Assessments in Large Language Models
• Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMs
• Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning
• LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation
• LLMs Don't Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanations
• Unbiased Reasoning for Knowledge-Intensive Tasks in Large Language Models via Conditional Front-Door Adjustment
• Clip Your Sequences Fairly: Enforcing Length Fairness for Sequence-Level RL
• Gradient-Attention Guided Dual-Masking Synergetic Framework for Robust Text-based Person Retrieval
• Benchmarking Energy Efficiency of Large Language Models Using vLLM
• Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
• Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
• Building Large-Scale English-Romanian Literary Translation Resources with Open Models
• TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modeling
• LongEmotion: Measuring Emotional Intelligence of Large Language Models in Long-Context Interaction
• Verbalized Algorithms: Classical Algorithms are All You Need (Mostly)
• Astra: A Multi-Agent System for GPU Kernel Performance Optimization
• DuoServe-MoE: Dual-Phase Expert Prefetch and Caching for LLM Inference QoS Assurance
• Toward Purpose-oriented Topic Model Evaluation enabled by Large Language Models
• COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens
• RAFFLES: Reasoning-based Attribution of Faults for LLM Systems
• VehicleWorld: A Highly Integrated Multi-Device Environment for Intelligent Vehicle Interaction
• Ban&Pick: Ehancing Performance and Efficiency of MoE-LLMs via Smarter Routing
• FineServe: Precision-Aware KV Slab and Two-Level Scheduling for Heterogeneous Precision LLM Serving
• MaaSO: SLO-aware Orchestration of Heterogeneous Model Instances for MaaS
• IPR: Intelligent Prompt Routing with User-Controlled Quality-Cost Trade-offs
• Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platforms
• From Long to Short: LLMs Excel at Trimming Own Reasoning Chains
• Mitigating Spurious Correlations Between Question and Answer via Chain-of-Thought Correctness Perception Distillation
• Self-Aligned Reward: Towards Effective and Efficient Reasoners
• PLaMo 2 Technical Report
• VoltanaLLM: Feedback-Driven Frequency Control and State-Space Routing for Energy-Efficient LLM Serving
• KVCompose: Efficient Structured KV Cache Compression with Composite Tokens
• Memorization ≠ Understanding: Do Large Language Models Have the Ability of Scenario Cognition?
• SpikingBrain: Spiking Brain-inspired Large Models
• PagedEviction: Structured Block-wise KV Cache Pruning for Efficient Large Language Model Inference
• MAGneT: Coordinated Multi-Agent Generation of Synthetic Multi-Turn Mental Health Counseling Sessions
• LibriQuote: A Speech Dataset of Fictional Character Utterances for Expressive Zero-Shot Speech Synthesis
• GreenLLM: SLO-Aware Dynamic Frequency Scaling for Energy-Efficient LLM Serving
• Code Like Humans: A Multi-Agent Solution for Medical Coding
• Cetvel: A Unified Benchmark for Evaluating Language Understanding, Generation and Cultural Capacity of LLMs for Turkish
• Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agents
• Retrieval-Augmented Defense: Adaptive and Controllable Jailbreak Prevention for Large Language Models
• MoPEQ: Mixture of Mixed Precision Quantized Experts
• Scalable hybrid quantum Monte Carlo simulation of U(1) gauge field coupled to fermions on GPU
• Attributes as Textual Genes: Leveraging LLMs as Genetic Algorithm Simulators for Conditional Synthetic Data Generation
• LExI: Layer-Adaptive Active Experts for Efficient MoE Model Inference
• Batch Query Processing and Optimization for Agentic Workflows
• ProST: Progressive Sub-task Training for Pareto-Optimal Multi-agent Systems Using Small Language Models
• Jointly Reinforcing Diversity and Quality in Language Model Generations
• Baichuan-M2: Scaling Medical Capability with Large Verifier System
• Bridging the Gap in Ophthalmic AI: MM-Retinal-Reason Dataset and OphthaReason Model toward Dynamic Multimodal Reasoning
• Learned Hallucination Detection in Black-Box LLMs using Token-level Entropy Production Rate
• CYCLE-INSTRUCT: Fully Seed-Free Instruction Tuning via Dual Self-Training and Cycle Consistency
• ABCD-LINK: Annotation Bootstrapping for Cross-Document Fine-Grained Links
• KoBLEX: Open Legal Question Answering with Multi-hop Reasoning
• LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving
• DaMoC: Efficiently Selecting the Optimal Large Language Model for Fine-tuning Domain Tasks Based on Data and Model Compression
• Natural Context Drift Undermines the Natural Language Understanding of Large Language Models
• FlashAdventure: A Benchmark for GUI Agents Solving Full Story Arcs in Diverse Adventure Games
• Street-Level Geolocalization Using Multimodal Large Language Models and Retrieval-Augmented Generation
• GPT-OSS-20B: A Comprehensive Deployment-Centric Analysis of OpenAI's Open-Weight Mixture of Experts Model
• DPF-CM: A Data Processing Framework with Privacy-Preserving Vector Databases for Chinese Medical LLMs Training and Deployment
• DSDE: Dynamic Speculative Decoding with KLD Stability for Real-World Serving
• Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward
• The Good, the Bad and the Constructive: Automatically Measuring Peer Review's Utility for Authors
• KVComp: A High-Performance, LLM-Aware, Lossy Compression Framework for KV Cache
• If We May De-Presuppose: Robustly Verifying Claims through Presupposition-Free Question Decomposition
• When Thinking Backfires: Mechanistic Insights Into Reasoning-Induced Misalignment
• ParaThinker: Native Parallel Thinking as a New Paradigm to Scale LLM Test-time Compute
• Scalable Option Learning in High-Throughput Environments
• FlexLink: Boosting your NVLink Bandwidth by 27% without accuracy concern
• Democratizing Agentic AI with Fast Test-Time Scaling on the Edge
• Do Cognitively Interpretable Reasoning Traces Improve LLM Performance?
• Automated Clinical Problem Detection from SOAP Notes using a Collaborative Multi-Agent LLM Architecture
• PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chains
• Stairway to Fairness: Connecting Group and Individual Fairness
• Know When to Explore: Difficulty-Aware Certainty as a Guide for LLM Reinforcement Learning
• Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learning
• AI Compute Architecture and Evolution Trends
• The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management
• MOSAIC: A Multilingual, Taxonomy-Agnostic, and Computationally Efficient Approach for Radiological Report Classification
• Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
• Improving Aviation Safety Analysis: Automated HFACS Classification Using Reinforcement Learning with Group Relative Policy Optimization
• HyperFlexis: Joint Design of Algorithms and Systems for Multi-SLO Serving and Fast Scaling
• Evaluating Structured Decoding for Text-to-Table Generation: Evidence from Three Datasets
• MSRS: Evaluating Multi-Source Retrieval-Augmented Generation
• Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning
• AWorld: Orchestrating the Training Recipe for Agentic AI
• Revealing Potential Biases in LLM-Based Recommender Systems in the Cold Start Setting
• End-to-End Agentic RAG System Training for Traceable Diagnostic Reasoning
• AdaptCache: KV Cache Native Storage Hierarchy for Low-Delay and High-Quality Language Model Serving
• TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
• cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending
• DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis
• SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Control
• HEAL: A Hypothesis-Based Preference-Aware Analysis Framework
• Secure Multi-LLM Agentic AI and Agentification for Edge General Intelligence by Zero-Trust: A Survey
• Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?
• Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
• Taming the Chaos: Coordinated Autoscaling for Heterogeneous and Disaggregated LLM Inference
• Alignment with Fill-In-the-Middle for Enhancing Code Generation
• Improving Low-Resource Translation with Dictionary-Guided Fine-Tuning and RL: A Spanish-to-Wayuunaiki Study
• Enabling Transparent Cyber Threat Intelligence Combining Large Language Models and Domain Ontologies
• HAP: Hybrid Adaptive Parallelism for Efficient Mixture-of-Experts Inference
• LongReasonArena: A Long Reasoning Benchmark for Large Language Models
• Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning
• ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitive
• Strata: Hierarchical Context Caching for Long Context Language Model Serving
• Rethinking Caching for LLM Serving Systems: Beyond Traditional Heuristics
• History Rhymes: Accelerating LLM Reinforcement Learning with RhymeRL
• Hermes 4 Technical Report
• FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel
• SEAM: Semantically Equivalent Across Modalities Benchmark for Vision-Language Models
• How Quantization Shapes Bias in Large Language Models
• SparK: Query-Aware Unstructured Sparsity with Recoverable KV Cache Channel Pruning
• Position Bias Mitigates Position Bias:Mitigate Position Bias Through Inter-Position Knowledge Distillation
• Transduction is All You Need for Structured Data Workflows
• Correctness-Guaranteed Code Generation via Constrained Decoding
• PepThink-R1: LLM for Interpretable Cyclic Peptide Optimization with CoT SFT and Reinforcement Learning
• MoEcho: Exploiting Side-Channel Attacks to Compromise User Privacy in Mixture-of-Experts LLMs
• Powering Job Search at Scale: LLM-Enhanced Query Understanding in Job Matching Systems
• A Distributed Learned Hash Table
• InPars+: Supercharging Synthetic Data Generation for Information Retrieval Systems
• Equinox: Holistic Fair Scheduling in Serving Large Language Models
• ProMed: Shapley Information Gain Guided Reinforcement Learning for Proactive Medical LLMs
• Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR
• Datarus-R1: An Adaptive Multi-Step Reasoning LLM for Automated Data Analysis
• Stands to Reason: Investigating the Effect of Reasoning on Idiomaticity Detection
• X-MoE: Enabling Scalable Training for Emerging Mixture-of-Experts Architectures on HPC Platforms
• Mitigating Hallucinations in Large Language Models via Causal Reasoning
• STEM: Efficient Relative Capability Evaluation of LLMs through Structured Transition Samples
• AgentMental: An Interactive Multi-Agent Framework for Explainable and Adaptive Mental Health Assessment
• SeamlessFlow: A Trainer Agent Isolation RL Framework Achieving Bubble-Free Pipelines via Tag Scheduling
• AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities
• Searching for Privacy Risks in LLM Agents via Simulation
• EgoCross: Benchmarking Multimodal Large Language Models for Cross-Domain Egocentric Video Question Answering
• Dataset Construction for Training LLM to Learn Analog Circuit Knowledge
• Meta-Metrics and Best Practices for System-Level Inference Performance Benchmarking
• Pruning Long Chain-of-Thought of Large Reasoning Models via Small-Scale Preference Optimization
• Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
• Amazon Nova AI Challenge -- Trusted AI: Advancing secure, AI-assisted software development
• LLMC+: Benchmarking Vision-Language Model Compression with a Plug-and-play Toolkit
• Teaching LLMs to Speak Spectroscopy
• Slow Tuning and Low-Entropy Masking for Safe Chain-of-Thought Distillation
• The Surprising Effectiveness of Membership Inference with Simple N-Gram Coverage
• NeuronTune: Fine-Grained Neuron Modulation for Balanced Safety-Utility Alignment in LLMs
• Shadow in the Cache: Unveiling and Mitigating Privacy Risks of KV-cache in LLM Inference
• CodeGrad: Integrating Multi-Step Verification with Gradient-Based LLM Refinement
• READER: Retrieval-Assisted Drafter for Efficient LLM Inference
• Compass-Thinker-7B Technical Report
• ASPD: Unlocking Adaptive Serial-Parallel Decoding by Exploring Intrinsic Parallelism in LLMs
• TopXGen: Topic-Diverse Parallel Data Generation for Low-Resource Machine Translation
• KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration
• MiGrATe: Mixed-Policy GRPO for Adaptation at Test-Time
• CPO: Addressing Reward Ambiguity in Role-playing Dialogue via Comparative Policy Optimization
• Profiling Large Language Model Inference on Apple Silicon: A Quantization Perspective
• Retrospective Sparse Attention for Efficient Long-Context Generation
• Towards Efficient and Practical GPU Multitasking in the Era of LLM
• Efficient Speculative Decoding for Llama at Scale: Challenges and Solutions
• WeChat-YATT: A Scalable, Simple, Efficient, and Production Ready Training Library
• A Data-driven ML Approach for Maximizing Performance in LLM-Adapter Serving
• Semantic Caching for Low-Cost LLM Serving: From Offline Learning to Online Adaptation
• ThinkTuning: Instilling Cognitive Reflections without Distillation
• MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models
• Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference
• Enhancing Small LLM Alignment through Margin-Based Objective Modifications under Resource Constraints
• CCFQA: A Benchmark for Cross-Lingual and Cross-Modal Speech and Text Factuality Evaluation
• Dynamic Benchmark Construction for Evaluating Large Language Models on Real-World Codes
• Schema Lineage Extraction at Scale: Multilingual Pipelines, Composite Evaluation, and Language-Model Benchmarks
• ALOPE: Adaptive Layer Optimization for Translation Quality Estimation using Large Language Models
• PrLM: Learning Explicit Reasoning for Personalized RAG via Contrastive Reward Optimization
• Pushing the Envelope of LLM Inference on AI-PC
• Inference-Aware Prompt Optimization for Aligning Black-Box Large Language Models
• Multimodal learning with next-token prediction for large multimodal models
• What Builds Effective In-Context Examples for Code Generation?
• Meta-Learning for Speeding Up Large Model Inference in Decentralized Environments
• LLM Serving Optimization with Variable Prefill and Decode Lengths
• Temporal Self-Rewarding Language Models: Decoupling Chosen-Rejected via Past-Future
• Devstral: Fine-tuning Language Models for Coding Agent Applications
• Mediator-Guided Multi-Agent Collaboration among Open-Source Models for Medical Decision-Making
• Impact-driven Context Filtering For Cross-file Code Completion
• Pruning the Unsurprising: Efficient LLM Reasoning via First-Token Surprisal
• Efficient Reasoning for Large Reasoning Language Models via Certainty-Guided Reflection Suppression
• ReasoningTrack: Chain-of-Thought Reasoning for Long-term Vision-Language Tracking
• Aligning LLMs on a Budget: Inference-Time Alignment with Heuristic Reward Models
• Attention Basin: Why Contextual Position Matters in Large Language Models
• Decoupling Understanding from Reasoning via Problem Space Mapping for Small-Scale Model Reasoning
• Agnostics: Learning to Code in Any Programming Language via Reinforcement with a Universal Learning Environment
• Persistent Instability in LLM's Personality Measurements: Effects of Scale, Reasoning, and Conversation History
• Multi-module GRPO: Composing Policy Gradients and Prompt Optimization for Language Model Programs
• IFDECORATOR: Wrapping Instruction Following Reinforcement Learning with Verifiable Rewards
• Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning
• Accelerating Scientific Discovery with Multi-Document Summarization of Impact-Ranked Papers
• An Entity Linking Agent for Question Answering
• CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward
• Agent Lightning: Train ANY AI Agents with Reinforcement Learning
• Block: Balancing Load in LLM Serving with Context, Knowledge and Predictive Scheduling
• FilBench: Can LLMs Understand and Generate Filipino?
• Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning
• Frontier: Simulating the Next Generation of LLM Inference Systems
• Toward a Trustworthy Optimization Modeling Agent via Verifiable Synthetic Data Generation
• Survey of Large Language Models in Extended Reality: Technical Paradigms and Application Frontiers
• Mamba-X: An End-to-End Vision Mamba Accelerator for Edge Computing Devices
• CTTS: Collective Test-Time Scaling
• xDeepServe: Model-as-a-Service on Huawei CloudMatrix384
• NaviMaster: Learning a Unified Policy for GUI and Embodied Navigation Tasks
• Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving
• MicroMix: Efficient Mixed-Precision Quantization with Microscaling Formats for Large Language Models
• ReflecSched: Solving Dynamic Flexible Job-Shop Scheduling via LLM-Powered Hierarchical Reflection
• AGFT: An Adaptive GPU Frequency Tuner for Real-Time LLM Inference Optimization
• CoCoA: Collaborative Chain-of-Agents for Parametric-Retrieved Knowledge Synergy
• ConfGuard: A Simple and Effective Backdoor Detection for Large Language Models
• WarriorMath: Enhancing the Mathematical Ability of Large Language Models with a Defect-aware Framework
• The Promise of RL for Autoregressive Image Editing
• Optimal Scheduling Algorithms for LLM Inference: Theory and Practice
• GLiDRE: Generalist Lightweight model for Document-level Relation Extraction
• MELAC: Massive Evaluation of Large Language Models with Alignment of Culture in Persian Language
• Quality-of-Service Aware LLM Routing for Edge Computing with Multiple Experts
• ITDR: An Instruction Tuning Dataset for Enhancing Large Language Models in Recommendations
• EMA Without the Lag: Bias-Corrected Iterate Averaging Schemes
• CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning tasks
• BAR Conjecture: the Feasibility of Inference Budget-Constrained LLM Services with Authenticity and Reasoning
• KLLM: Fast LLM Inference with K-Means Quantization
• Where to show Demos in Your Prompt: A Positional Bias of In-Context Learning
• From Sufficiency to Reflection: Reinforcement-Guided Thinking Quality in Retrieval-Augmented Reasoning for LLMs
• BigTokDetect: A Clinically-Informed Vision-Language Modeling Framework for Detecting Pro-Bigorexia Videos on TikTok
• NeedleChain: Measuring Intact Long-Context Reasoning Capability of Large Language Models
• Hierarchical Verification of Speculative Beams for Accelerating LLM Inference
• Transformer (deep learning) [wikipedia]
• PagedAttention [wikipedia]
• VLLM [wikipedia]

Discussions

• Efficient Memory Management for Large Language Model Serving with PagedAttention [hn, 102 points, 16 comments]
• I found the following paper fascinating to read: it details PagedAttention, a novel paging-inspired way to slash memory waste and boost LLM serving throughput 2–4×. See link below. #LLM #AIinfrastruct [bsky, 7 points, 2 comments]
• Efficient Memory Management for Large Language Model Serving with PagedAttention [hn, 2 points, 0 comments]
• feeling exhausted but also a dopamine hit cuz, I finished the paper "Efficient Memory Management for LLMs Serving with Paged Attention." It took a lot of time for me to understand and recreate some [bsky, 1 points, 0 comments]

Related

• PagedEviction: Structured Block-wise KV Cache Pruning for Efficient Large Language Model Inference
• Injecting Adrenaline into LLM Serving: Boosting Resource Utilization and Throughput via Attention Disaggregation
• FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
• IC-Cache: Efficient Large Language Model Serving via In-context Caching
• Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving