Sign languages are visual-spatial languages, articulated not only by hands but also with facial expressions and other body parts. Facial expressions, in particular, have important grammatical roles, such as defining whether a sentence is a question or not and differentiating the meaning of a sign when assuming the role of a quantifier. Despite the importance of facial expressions, technologies to recognize and generate sign language mainly focus on manual signs, ignoring the information from facial expressions. In this work, we focus on collecting and annotating data to evaluate the role of facial expressions in communicating adjectives. We study the role of facial muscle activity to express the intensities of manual signs and show how modeling facial action units change the overall meaning of a sentence. We also explore how facial expressions allow a deeper understanding of sign language when included in the design of machine learning models.

End-to-end sign language understanding and generation models do not accurately represent the prosody of the languages. This lack of temporal and spatial variation in generated signs leads to poor quality and lower human perception. We seek to improve prosody in automatic models that understand sign language by modeling intensification in a data-driven manner with strategies grounded in the linguistics of sign language by enhancing the representation of intensity modifiers in gloss annotations. In this work, we identify four major categories for sign intensification across two different sign languages. Through an extensive data exploration, we compare annotated datasets for German Sign Language and British Sign Language for intensification. We analyze the effects of the domain of the dataset on the intensifier abundance and the changes of intensifier use across different sign languages.