Proceedings of the Annual Meeting of the Cognitive Science Society

Many non-native English speakers study mathematics in English, so they must learn both the math and the academic language used to express it. Large language models (LLMs) might help by producing worked solutions while also explaining difficult words and phrases, but it is unclear whether doing both harms mathematical accuracy. We test whether adding inline vocabulary explanations changes solution correctness. Using the CEFR-J vocabulary profile, we identify terms likely to need clarification and evaluate four open LLMs (2.7B–20B parameters) on English math problems in two conditions: standard solving vs. solving with embedded vocabulary support. On MMLU Elementary Mathematics and GSM8K, vocabulary scaffolding consistently reduces accuracy, with drops up to 16.2 percentage points. These findings reveal a trade-off between language assistance and reasoning performance. Educational interfaces may need to balance these goals carefully or separate language support from problem solving for English-medium instruction.