Proceedings of the Annual Meeting of the Cognitive Science Society

The present study compares human acceptability judgements of Korean clausal constructions—dative alternations, active–passive voice under animacy manipulation, and negative polarity item licensing—with surprisal estimates derived from GPT-style language models. Fifty-six native speakers evaluated 224 sentences, and surprisal was computed using three Korean-capable variants (KoGPT-2, Ko-GPT-Trinity, and mGPT). The models reproduced the strong preference for Dative–Accusative datives, captured voice- and animacy-related effects only partially, and failed to represent clause-bounded NPI licensing. These findings indicate limitations of surprisal as a proxy for acceptability beyond morphosyntax and motivate the development of cross-linguistic benchmarks for explainable AI and diagnostic evaluation.