Skip to main content
eScholarship
Open Access Publications from the University of California

Learning cognitive and linguistic prosodic categories for automatic cross-lingual sign language understanding

Creative Commons 'BY' version 4.0 license
Abstract

End-to-end sign language understanding and generation models do not accurately represent the prosody of the languages. This lack of temporal and spatial variation in generated signs leads to poor quality and lower human perception. We seek to improve prosody in automatic models that understand sign language by modeling intensification in a data-driven manner with strategies grounded in the linguistics of sign language by enhancing the representation of intensity modifiers in gloss annotations. In this work, we identify four major categories for sign intensification across two different sign languages. Through an extensive data exploration, we compare annotated datasets for German Sign Language and British Sign Language for intensification. We analyze the effects of the domain of the dataset on the intensifier abundance and the changes of intensifier use across different sign languages.

Main Content
For improved accessibility of PDF content, download the file to your device.
Current View