2025/06/10 by Issa Sugiura, Sugiura, Issa, Takashi Ishida +12 · 1 voice · 1 citation
Business, Management and Accounting · Computer Science · Decision Sciences · Economics, Econometrics and Finance · #Auditing, Earnings Management, Governance #Computation and Language (cs.CL) #Computational Engineering #FOS: Computer and information sciences #FOS: Economics and business #Finance #Financial Distress and Bankruptcy Prediction #Machine Learning (cs.LG) #Statistical Finance (q-fin.ST) #Stock Market Forecasting Methods #and Science (cs.CE) #cs.CE #cs.CL #cs.LG #q-fin.ST
paper · pdf · doi:10.48550/arxiv.2506.08762
openalex publication_date 2025/06/10 · arxiv published 2025/06/10 · openalex created_date 2025/10/10 · arxiv updated 2026/03/05 · openalex updated_date 2026/07/28
Large Language Models (LLMs) have made remarkable progress, surpassing human performance on several benchmarks in domains such as mathematics and coding. A key driver of this progress has been the development of benchmark datasets. In contrast, the financial domain poses higher entry barriers due to its demand for specialized expertise, and benchmarks remain relatively scarce compared to those in mathematics or coding. We introduce EDINET-Bench, an open-source Japanese financial benchmark designed to evaluate LLMs on challenging tasks such as accounting fraud detection, earnings forecasting, and industry classification. EDINET-Bench is constructed from ten years of annual reports filed by Japanese companies. These tasks require models to process entire annual reports and integrate information across multiple tables and textual sections, demanding expert-level reasoning that is challenging even for human professionals. Our experiments show that even state-of-the-art LLMs struggle in this domain, performing only marginally better than logistic regression in binary classification tasks such as fraud detection and earnings forecasting. Our results show that simply providing reports to LLMs in a straightforward setting is not enough. This highlights the need for benchmark frameworks that better reflect the environments in which financial professionals operate, with richer scaffolding such as realistic simulations and task-specific reasoning support to enable more effective problem solving. We make our dataset and code publicly available to support future research.