Accounting Reasoning in Large Language Models: Concepts, Evaluation, and Empirical Analysis

2025/12/27 by Jie Zhou, Zhou, Jie, Xin Chen +5
Computer Science · Materials Science · Social Sciences · #Computation and Language (cs.CL) #Computational and Text Analysis Methods #FOS: Computer and information sciences #Machine Learning in Materials Science #Topic Modeling

paper · doi:10.48550/arxiv.2512.22443

openalex publication_date 2025/12/27 · openalex created_date 2025/12/31 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) are increasingly reshaping learning paradigms, cognitive processes, and research methodologies across diverse domains. As their adoption expands, effectively integrating LLMs into professional fields and clarifying their role in domain-specific applications has become a key challenge for enterprise digital transformation and broader societal development. In the accounting domain, successful integration requires a systematic understanding of LLMs' domain-specific reasoning capabilities. In this study, we introduce the concept of accounting reasoning and propose a set of evaluation criteria grounded in an analysis of the training data characteristics of representative GLM-series models. These criteria establish a foundation for studying accounting-oriented reasoning paradigms and provide benchmarks for assessing and improving model performance. Building on this framework, we evaluate several representative LLMs, including GLM-6B, GLM-130B, GLM-4, and GPT-4, across a range of accounting reasoning tasks. Our experimental results show that prompt engineering strategies can yield varying degrees of performance improvement across models, with GPT-4 demonstrating the strongest overall accounting reasoning capability. Nevertheless, the results indicate that current LLMs remain insufficient for real-world accounting applications. In particular, further optimization is required for deployment in enterprise-level accounting scenarios to fully realize the potential value of LLMs in this domain.

Citations

• Deliberate Reasoning in Language Models as Structure-Aware Planning with an Accurate World Model
• TILP: Differentiable Learning of Temporal Logical Rules on Knowledge Graphs
• Evaluating LLMs' Mathematical Reasoning in Financial Document Question Answering
• Large Language Models Can Learn Temporal Reasoning
• MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation
• TEILP: Time Prediction over Knowledge Graphs via Logical Reasoning
• NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes
• FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets
• From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning
• Harnessing the Power of Large Language Models for Natural Language to First-Order Logic Translation
• BloombergGPT: A Large Language Model for Finance
• CLUE: A Chinese Language Understanding Evaluation Benchmark
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

Related

• Large Language Models Acing Chartered Accountancy
• FinEval-KR: A Financial Domain Evaluation Framework for Large Language Models' Knowledge and Reasoning
• Evaluating Large Language Models for Financial Reasoning: A CFA-Based Benchmark Study
• Beyond Classification: Financial Reasoning in State-of-the-Art Language Models
• Assessing Consistency and Reproducibility in the Outputs of Large Language Models: Evidence Across Diverse Finance and Accounting Tasks