EnviroLLM: Resource Tracking and Optimization for Local AI

2025/12/12 by Troy R. Allen, Allen, Troy
Computer Science · Decision Sciences · #Computers and Society (cs.CY) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Scientific Computing and Data Management #Software System Performance and Reliability

paper · pdf · doi:10.48550/arxiv.2512.12004

openalex publication_date 2025/12/12 · openalex created_date 2025/12/17 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) are increasingly deployed locally for privacy and accessibility, yet users lack tools to measure their resource usage, environmental impact, and efficiency metrics. This paper presents EnviroLLM, an open-source toolkit for tracking, benchmarking, and optimizing performance and energy consumption when running LLMs on personal devices. The system provides real-time process monitoring, benchmarking across multiple platforms (Ollama, LM Studio, vLLM, and OpenAI-compatible APIs), persistent storage with visualizations for longitudinal analysis, and personalized model and optimization recommendations. The system includes LLM-as-judge evaluations alongside energy and speed metrics, enabling users to assess quality-efficiency tradeoffs when testing models with custom prompts.

Citations

• The Hidden Costs of Translation Accuracy: Distillation, Quantization, and Environmental Impact
• Ground-Truthing AI Energy Consumption: Validating CodeCarbon Against External Measurements
• Accurate and Energy Efficient: Local Retrieval-Augmented Generation Models Outperform Commercial Large Language Models in Medical Tasks
• Sustainability via LLM Right-sizing
• LLM-as-a-Judge: Reassessing the Performance of LLMs in Extractive QA
• Optimizing Large Language Models: Metrics, Energy Efficiency, and Case Study Insights
• Carbon Footprint Evaluation of Code Generation through LLM as a Service
• OpenCarbonEval: A Unified Carbon Emission Estimation Framework in Large-Scale AI Models
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Quantifying the Carbon Emissions of Machine Learning

Related

• Benchmarking Energy Efficiency of Large Language Models Using vLLM
• How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference
• Optimizing Large Language Models: Metrics, Energy Efficiency, and Case Study Insights
• MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Cases
• Evaluating the Limitations of Local LLMs in Solving Complex Programming Challenges