2025/03/06 by Niyar R Barman, Sharma, Krish, Barman, Niyar R +4 · 1 citation
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Mathematics, Computing, and Information Processing #Text Readability and Simplification #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2503.04685
openalex publication_date 2025/03/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We look at reasoning on GSM8k, a dataset of short texts presenting primary school, math problems. We find, with Mirzadeh et al. (2024), that current LLM progress on the data set may not be explained by better reasoning but by exposure to a broader pretraining data distribution. We then introduce a novel information source for helping models with less data or inferior training reason better: discourse structure. We show that discourse structure improves performance for models like Llama2 13b by up to 160%. Even for models that have most likely memorized the data set, adding discourse structural information to the model still improves predictions and dramatically improves large model performance on out of distribution examples.