TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task
2025/07/22 by Michael R. Bock, Bock, Michael R., Kara Molisee +5 · 12 voices
#cs.AI
paper · pdf · doi:10.48550/arxiv.2507.16126
Abstract
Can AI file your taxes? Not yet. Calculating US personal income taxes is a task that requires building an understanding of vast amounts of English text and using that knowledge to carefully compute results. We propose TaxCalcBench, a benchmark for determining models' abilities to calculate personal income tax returns given all of the necessary information. Our experiment shows that state-of-the-art models succeed in calculating less than a third of federal income tax returns even on this simplified sample set. Our analysis concludes that models consistently misuse tax tables, make errors in tax calculation, and incorrectly determine eligibility. Our findings point to the need for additional infrastructure to apply LLMs to the personal income tax calculation task.
Discussions
- TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task [hn, 70 points, 24 comments]
- many benchmarks used to measure AI capabilities are, I think, contrived and lenient. here's a good real-life study, on whether AI can do your (US) tax returns; a domain with plentiful training data an [bsky, 26 points, 3 comments]
- TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task [hn, 2 points, 0 comments]
- TaxCalcBench: Can AI file your taxes? (not yet) [hn, 1 points, 0 comments]
- https://bsky.app/profile/ruoshuiresearch.bsky.social/post/3lvnqr3h6ls26 [bsky, 1 points, 1 comments]
- Steuern mit KI statt Steuerberater? Wenn du nicht im Knast landen willst, vielleicht eher nicht: "Our experiment shows that state-of-the-art models succeed in calculating less than a third of federal [bsky, 0 points, 0 comments]
- TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task https://arxiv.org/abs/2507.16126 (https://news.ycombinator.com/item?id=45601230) [bsky, 0 points, 0 comments]
- 2/ Today, we’re releasing TaxCalcBench: a first-ever benchmark dataset & eval framework for testing AI’s ability to calculate US personal income tax returns. Tax is a secretive industry, so we’re prou [bsky, 0 points, 1 comments]
- TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task [bsky, 0 points, 0 comments]
- TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task https://arxiv.org/abs/2507.16126 (https://news.ycombinator.com/item?id=45601230) [bsky, 0 points, 0 comments]
- TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task https://arxiv.org/abs/2507.16126 [bsky, 0 points, 0 comments]
- TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task https://arxiv.org/abs/2507.16126 [comments] [32 points] [bsky, 0 points, 0 comments]
Related