ProfInfer: An eBPF-based Fine-Grained LLM Inference Profiler

2026/01/28 by Bohua Zou, Debayan Roy, Dhimankumar Yogesh Airao +4 · 2 voices
Computer Science · #Natural Language Processing Techniques #Software System Performance and Reliability #Topic Modeling #cs.SE

paper · pdf · doi:10.48550/arxiv.2601.20755

openalex publication_date 2026/01/28 · arxiv published 2026/01/28 · arxiv updated 2026/01/29 · openalex created_date 2026/01/30 · openalex updated_date 2026/07/28

Abstract

As large language models (LLMs) move from research to production, understanding how inference engines behave in real time has become both essential and elusive. Unlike general-purpose engines such as ONNX Runtime, today's LLM inference systems offer little operator-level visibility, leaving developers blind to where time and resources go. Even basic questions -- is this workload memory-bound or compute-bound? -- often remain unanswered. To close this gap, we develop a fine-grained, non-intrusive profiling framework for modern LLM inference engines, exemplified by llama-cpp but applicable to similar runtime architectures. Built on extended Berkeley Packet Filter (eBPF) technology, our system dynamically attaches probes to runtime functions across multiple layers -- without modifying or recompiling the source. It transforms collected traces into rich visualizations of operators, graphs, timelines, and hardware counter trends, exposing how dense inference, Mixture-of-Experts routing, and operator offloading behave in practice. With less than 4% runtime overhead and high profiling fidelity, our framework makes LLM inference both transparent and diagnosable, turning performance profiling into a practical tool for optimization, scheduling, and resource-aware deployment.

Citations

• Characterizing Mobile SoC for Accelerating Heterogeneous LLM Inference
• MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices
• PalmBench: A Comprehensive Benchmark of Compressed Large Language Models on Mobile Platforms
• The Llama 3 Herd of Models
• Fast On-device LLM Inference with NPUs
• PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU
• Trace-enabled Timing Model Synthesis for ROS2-based Autonomous Applications
• Qwen Technical Report
• Efficient Memory Management for Large Language Model Serving with PagedAttention
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
• Fast Inference from Transformers via Speculative Decoding
• GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
• MNN: A Universal and Efficient Inference Engine
• Attention Is All You Need

Discussions

• ProfInfer: An eBPF-based Fine-Grained LLM Inference Profiler arxiv.org/abs/2601.20755 [bsky, 2 points, 0 comments]
• With many LLM being black boxes, developers are left asking basic questions: Is this workload memory-bound or compute-bound? And getting no answers. Enter eBPF and ProfInfer from Bohua Zou to shed lig [bsky, 1 points, 0 comments]

Related

• Vidur: A Large-Scale Simulation Framework For LLM Inference
• UELLM: A Unified and Efficient Approach for LLM Inference Serving
• Bench360: Benchmarking Local LLM Inference from 360 Degrees
• iServe: An Intent-based Serving System for LLMs
• LLM Inference Unveiled: Survey and Roofline Model Insights