This paper on cognitive complexity in primates and cetaceans is a review of studies that use onlyobservational methods. These studies include descriptive accounts, both qualitative and quantitative,of behavior-in-context in naturally-occurring and quasi-experimental settings, especially involvingthe micro-analysis of video. To unify this piecemeal but burgeoning literature, “cognition” is taken asembodied, largely visible, and distributed across physical and social environments. Its study involvesdocumenting the adaptation of behavior to changing conditions, especially in ontogeny, tool-use, andsocial discourse. The studies selected for this review focus on the cognitive complexity that isapparent in the versatility, the hierarchical organization, and the long-term patterning of suchbehavioral adaptations. Versatility is seen, for example, in the substitution of different acts or objectsinto established routines, in the size and flexibility of action repertoires that enable variablyconfigured and sequenced performances, and in the marked occurrence of individual differences.Hierarchical organization is seen in the substitution or iteration of a subroutine that fails to disrupt itslarger routine, in the simultaneous embedding of one social interaction within the frame of another(as in “social tool” use), and in the insertion of a novel or borrowed subroutine as a tactical response,especially one that temporarily redirects an animal‟s trajectory. The complexity apparent in long-termpatterning includes tracking and making selective use of multiple histories (e.g., concerning kinship,rank, etc.) whose predictions and tactics may vary, responding to “market” values that change withecological and social factors, and exploiting traditions of practice which provide social and materialresources that shape engagement and learning. While this literature includes far more primate thancetacean examples, the primate work offers helpful suggestions for settings, issues, and techniquesthat could be adapted to the sensori-motor, ecological, and social constraints on cetacean cognition.The array of observations reviewed illustrate the utility across species of scoring such parameters asdisplays of attention in multiple modalities, abrupt trajectory changes, the complementarity andcontingency of actions, and the resiliency of sequences, to help identify the media that matter in agiven cognitive ecology. Systematic micro-analyses, in conjunction with long-term relational datathat track changes in affordances and coordination, make such observational approaches a viable andvaluable addition to the study of comparative cognition.

The “S-posture” is described in the cetacean literature as a radical flexure of the body which presents an atypically vertical visual signal. It has most commonly been associated with agonistic high arousal contexts, and often includes simultaneous acoustic outbursts. Its dynamic qualities – an abrupt retardation of forward motion, sweeping flexure of the flukes, and sustained arch – suggest its saliency to the cetacean’s motion-sensitive visual system. This study reports on the occurrence of S-postures in four captive beluga whales ( Delphinapterus leucas ) held at SeaWorld San Diego. During approximately 27 hours of video data, a total of 174 S-postures were displayed by three out of four belugas. None of the S-postures observed co-occurred with another visual display (i.e., bubbleclouds, open mouth, jaw clap), while only 8% were observed to have co-occurred with an acoustic production by the whales present. The proportion of S-postures displayed by each subject was analyzed for differences in the following contexts: the state (open/closed) of a rear gate leading to a separate pool, the presence of cohabitant harbor seals ( Phoca vitulina ), and the total number ofbelugas in the same pool.

"Gaze following"-when one individual witnesses another shift its orientation, and then re-orients in the same direction-has been observed in a wide range of species. Related work with dolphins has to date focused on human-dolphin interactions. In this conspecific study, we examined a group of dolphins orienting, in passing, to gateways between their pools, as opportunities for witnesses to demonstrate "gaze following". Seven bottlenose dolphins were synchronously videotaped on six underwater cameras, for 21 h over three days, and the recordings analyzed by trained observers. The identities of all animals present, their partner state, and whether and to what degree they had altered their access to the gate (e.g., from Monocular to Binocular, or Binocular to Visio-Echoic) was recorded. Compared to animals that did not witness such a change, witnesses of an increase in access by another dolphin were significantly more likely to also act to increase their own access. We observed 460 such cases of "gaze following" in these animals. Dolphins who were partnered (showed sustained swimming within 1 body length) were significantly more likely, than non-partnered animals, to "gaze follow". Dolphins also showed a significant tendency toward matching the kind of access they observed. No significant difference was found in the presence of animals in the back pools, during changes in orientation that were followed, versus in those that were not. These findings support adding bottlenose dolphins to the growing list of species that display conspecific "gaze following".

Previous developmental accounts of joint object activity identify a qualitative "shift" around 9-12 months. In a longitudinal study of 26 dyads, videos of joint object interactions at 4, 6, 9, and 12 months were coded for all targets of gaze and manual activity (at 10 Hz). At 12 months, infants distribute their sensorimotor modalities between objects handled by the parent and others controlled by the infant. Analyses reveal novel trajectories in distributed joint object activity across the 1st year. At 4 months, infants predominantly look at and manipulate a single object, typically held by their mothers. Between 6 and 9 months, infants increasingly decouple their visual and haptic modalities and distribute their attention between objects held by their mothers and by themselves. These previously unreported developments in the distribution of multimodal object activity might "bridge the gap" to coordinated joint activity between 6 and 12 months.

This project characterizes the development of early social interactions in a longitudinal sample of mothers and their infants. To do this we track the interaction in as a temporal configuration of multimodal, multiparty components, coding a total of 14 dimensions at each frame. Our final goal is to develop methods which we can use to systematically document regularities in the way attention gets organized longitudinally in a large sample of mother-infant dyads. Coding systematically at this scale allows us to ground a developmental account of early social interaction. For this project, we developed 1) ways to reduce the labor intensity of coding frame-by-frame multidimensional data over the entire sample and 2)ways of quantifying complex configural regularities across our sample. © ACM 2010.