What GPT language models get right—and wrong—about honorific expression: Evidence from Korean subject honorification
Abstract
The present study investigates how GPT language models approximate human sentence-processing dynamics observed in Korean subject honorification. Drawing upon two open-access datasets, we compare human responses with surprisal estimates from two Korean-capable GPT models. Across tasks, the models reliably identified overt morphosyntactic violations, assigning high surprisal to clear honorific agreement mismatches associated with low politeness ratings and increased reading times in humans. However, GPT surprisal failed to capture more graded and context-sensitive effects, including the optionality of the subject honorific suffix, animacy-dependent politeness evaluations, and delayed spillover effects in human reading. Correlations between human measures and model surprisal were consistently weak and highly condition-specific. Taken together, while GPT surprisal serves as a coarse indicator of well-formedness, it inadequately represents the socio-pragmatic and discourse-level constraints that shape Korean sentence processing. This calls for the need for more cognitively and culturally grounded metrics in language sciences and explainable AI.