UC Santa Barbara

Interpretation of gaze direction is an essential ability that allows humans to infer others' intentions, interests, and future actions in daily social interactions. Humans start to respond to direct and avert gazes at 4 months old (Farroni et al., 2002), and show the ability to follow others’ gaze direction at 10 months old (Brooks & Meltzoff, 2005). Studies have shown that gaze perception relies on the integrated effect of the eye region, head orientation, and body posture (Cline, 1967, p. 19; Hietanen, 2002; Langton et al., 2004; Moors et al., 2015; Otsuka, 2014; Pomianowska et al., 2012; Wollaston, 1824). The gaze of others influences both covert (Bayliss et al., 2004a; Driver et al., 1999; Friesen & Kingstone, 1998) and overt attention (Friesen & Kingstone, 2003; Hood et al., 1998; Mansfield et al., 2003; Ricciardelli et al., 2003). Not only eye gaze but head and body posture also orient observers’ attention (Azarian et al., 2017; Bayliss et al., 2004a; Frischen et al., 2007, p. 20; Hietanen, 1999, 2002). Studies have argued that images of gaze induce attention shift exogenously because the attention orientation effect following the gaze direction is preserved even when the gaze direction is uninformative (Driver et al., 1999; Friesen et al., 2004; Friesen & Kingstone, 1998; Ristic et al., 2007). However, unlike typical peripheral exogenous cues (e.g., flashlight) that only sustain a transient attention orientation that lasts less than 200ms (Posner & Cohen, 1984; Theeuwes, 1991a), attention orientation elicited from gaze develops gradually and decays after 500ms since the cue onset (Friesen & Kingstone, 1998; McKee et al., 2007; Müller & Findlay, 1988a).A limitation of the majority of gaze studies is that they typically use simple images of gaze, foveally presented, and include a simple perceptual task. These displays do not incorporate the dynamics of head and body movements, the surrounding real-world contexts in scenes, or more ecological tasks. Less is known about how humans estimate gaze, orient covert attention, and eye movements with real-world scenes that include the dynamics of gazer movements. The moment-to-moment control of eye movements during gaze-following has also not been studied. The overall goal of this dissertation is to increase the understanding of the perception of gaze and the influence of gaze on attention and eye movements in real-world scenes. Chapter I studies how humans and the state-of-art AI model perceive gaze in naturalistic gaze images. By digitally manipulating the contextual information in the scene (the number of gazers and the presence of the gazed target), Chapter I shows that humans have a higher sensitivity to contextual information changes compared to the AI model, and a potential direction for developing AI model with better ability to integrate spatial information in the future. Chapter II-III explores the process of covert attention orientation, microsaccades bias, and overt attention (eye movement) using videos of dynamic gazer behaviors. Both chapters show the importance of head dynamics in orienting attention. Specifically, Chapter II illustrates the importance of the presence of both head and body motion to sustain covert attention for a longer period (500ms), and microsaccades are correlated with behavioral performance. Chapter III shows that valid gaze cues guide eye movements faster and more accurately toward the gazed target (< 2˚) and improve behavioral performance. It also suggests that the smallest gaze cue guidance effect when heads are removed may be due to the reduced available information in headless bodies. Lastly, Chapter IV proceeds to quantify the dynamic changes of gazer’s heads and relate them to the dynamics of eye movements during gaze following. It demonstrates that observers use information in the visual periphery to execute predictive saccades that anticipate the information in the gazer’s head direction by 190-350ms. Observers simultaneously monitor moment-to-moment changes in the gazer’s head velocity to dynamically alter eye movements and re-fixate the gazer (reverse saccades) when the head accelerates before the initiation of the first forward gaze-following saccade. Together, the dissertation extends our understanding of gaze perception in real-life contexts, and its impacts on observers’ covert attention, overt attention, and fine-grained eye movement control with ecologically relevant dynamic gaze behaviors.