Proceedings of the Annual Meeting of the Cognitive Science Society

Analyzing how human beings resolve syntactic ambiguity haslong been an issue of interest in the field of linguistics. It is, atthe same time, one of the most challenging issues for spokenlanguage understanding (SLU) systems as well. As syntacticambiguity is intertwined with issues regarding prosody and se-mantics, the computational approach toward speech intentionidentification is expected to benefit from the observations ofthe human language processing mechanism. In this regard, weaddress the task with attentive recurrent neural networks thatexploit acoustic and textual features simultaneously and revealhow the modalities interact with each other to derive sentencemeaning. Utilizing a speech corpus recorded on Korean scriptsof syntactically ambiguous utterances, we revealed that co-attention frameworks, namely multi-hop attention and cross-attention, show significantly superior performance in disam-biguating speech intention. With further analysis, we demon-strate that the computational models reflect the internal rela-tionship between auditory and linguistic processes.