Temperature in SLMs: Impact on Incident Categorization in On-Premises Environments

2025/11/21 by Pohlmann, Marcio, Severo, Alex, Almeida, Gefté +3
#68T01 #Artificial Intelligence (cs.AI) #Cryptography and Security (cs.CR) #Distributed #FOS: Computer and information sciences #I.2 #Machine Learning (cs.LG) #Parallel #Performance (cs.PF) #and Cluster Computing (cs.DC)

paper · doi:10.48550/arxiv.2511.19464

Abstract

SOCs and CSIRTs face increasing pressure to automate incident categorization, yet the use of cloud-based LLMs introduces costs, latency, and confidentiality risks. We investigate whether locally executed SLMs can meet this challenge. We evaluated 21 models ranging from 1B to 20B parameters, varying the temperature hyperparameter and measuring execution time and precision across two distinct architectures. The results indicate that temperature has little influence on performance, whereas the number of parameters and GPU capacity are decisive factors.

Citations

• Exploring the Impact of Temperature on Large Language Models:Hot or Cold?
• A Survey of Large Language Models
• Contextual Temperature for Language Modeling

Cited by

• On-Premise SLMs vs. Commercial LLMs: Prompt Engineering and Incident Classification in SOCs and CSIRTs

Related

• TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms
• Confidential LLM Inference: Performance and Cost Across CPU and GPU TEEs
• WarmServe: Enabling One-for-Many GPU Prewarming for Multi-LLM Serving
• iServe: An Intent-based Serving System for LLMs
• Cooling Matters: Benchmarking Large Language Models and Vision-Language Models on Liquid-Cooled Versus Air-Cooled H100 GPU Systems