2024/04/12 by Xiang Zhang, Zhang, Xiang, Khatoon Khedri +3
Computer Science · Decision Sciences · #68-04 #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Data Quality and Management #Databases (cs.DB) #FOS: Computer and information sciences #H.2.m #Software Engineering Research #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2404.08727
openalex publication_date 2024/04/12 · openalex created_date 2024/04/17 · openalex updated_date 2026/07/28
Large Language Models (LLMs) can automate or substitute different types of tasks in the software engineering process. This study evaluates the resource utilization and accuracy of LLM in interpreting and executing natural language queries against traditional SQL within relational database management systems. We empirically examine the resource utilization and accuracy of nine LLMs varying from 7 to 34 Billion parameters, including Llama2 7B, Llama2 13B, Mistral, Mixtral, Optimus-7B, SUS-chat-34B, platypus-yi-34b, NeuralHermes-2.5-Mistral-7B and Starling-LM-7B-alpha, using a small transaction dataset. Our findings indicate that using LLMs for database queries incurs significant energy overhead (even small and quantized models), making it an environmentally unfriendly approach. Therefore, we advise against replacing relational databases with LLMs due to their substantial resource utilization.