ProfilingAgent: Profiling-Guided Agentic Reasoning for Adaptive Model Optimization
2025/09/06 by Sadegh Jafari, Aishwarya Sarkar, Jafari, Sadegh +5
Business, Management and Accounting · Computer Science · Decision Sciences · #Business Process Modeling and Analysis #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multi-Agent Systems and Negotiation #Performance (cs.PF) #Simulation Techniques and Applications
paper · pdf · doi:10.48550/arxiv.2509.05584
openalex publication_date 2025/09/06 · openalex created_date 2025/10/11 · openalex updated_date 2026/07/28
Abstract
Foundation models face growing compute and memory bottlenecks, hindering deployment on resource-limited platforms. While compression techniques such as pruning and quantization are widely used, most rely on uniform heuristics that ignore architectural and runtime heterogeneity. Profiling tools expose per-layer latency, memory, and compute cost, yet are rarely integrated into automated pipelines. We propose ProfilingAgent, a profiling-guided, agentic approach that uses large language models (LLMs) to automate compression via structured pruning and post-training dynamic quantization. Our modular multi-agent system reasons over static metrics (MACs, parameter counts) and dynamic signals (latency, memory) to design architecture-specific strategies. Unlike heuristic baselines, ProfilingAgent tailors layer-wise decisions to bottlenecks. Experiments on ImageNet-1K, CIFAR-10, and CIFAR-100 with ResNet-101, ViT-B/16, Swin-B, and DeiT-B/16 show pruning maintains competitive or improved accuracy (about 1% drop on ImageNet-1K, +2% gains for ViT-B/16 on smaller datasets), while quantization achieves up to 74% memory savings with <0.5% accuracy loss. Our quantization also yields consistent inference speedups of up to 1.74 times faster. Comparative studies with GPT-4o and GPT-4-Turbo highlight the importance of LLM reasoning quality for iterative pruning. These results establish agentic systems as scalable solutions for profiling-guided model optimization.
Citations
- MergeQuant: Accurate 4-bit Static Quantization of Large Language Models by Channel-wise Calibration
- SuperSAM: Crafting a SAM Supernetwork via Structured Pruning and Unstructured Parameter Prioritization
- ShortGPT: Layers in Large Language Models are More Redundant Than You Expect
- Comparative Analysis of CPU and GPU Profiling for Deep Learning Models
- A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations
- A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations
- Pruning vs Quantization: Which is Better?
- Temporal Dynamic Quantization for Diffusion Models
- SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks
- LLM-Pruner: On the Structural Pruning of Large Language Models
- Generative Agents: Interactive Simulacra of Human Behavior
- Toolformer: Language Models Can Teach Themselves to Use Tools
- Distributing Accountability, Not Capability: Phase Separation and the LLM Workflow Quadrant in Autonomous AI Agent Architectures
- Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango
- NIPQ: Noise proxy-based Integrated Pseudo-Quantization
- A Generalist Agent
- INSTA-BNN: Binary Neural Network with INSTAnce-aware Threshold
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
- STaR: Bootstrapping Reasoning With Reasoning
- QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization
- Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks
- BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
- A Survey of Quantization Methods for Efficient Neural Network Inference
- BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction
- Training data-efficient image transformers & distillation through attention
- Up or Down? Adaptive Rounding for Post-Training Quantization
- LSQ+: Improving low-bit quantization through learnable offsets and better initialization
- Training with Quantization Noise for Extreme Model Compression
- Implicit Filter Sparsification In Convolutional Neural Networks
- The State of Sparsity in Deep Neural Networks
- Learned Step Size Quantization
- Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration
- Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss
- PACT: Parameterized Clipping Activation for Quantized Neural Networks
- Pruning Filters for Efficient ConvNets
- Deep Residual Learning for Image Recognition
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Related